DiscoverSecurity Unlocked
Security Unlocked
Claim Ownership

Security Unlocked

Author: Microsoft

Subscribed: 59Played: 415


Security Unlocked explores the technology and people powering Microsoft's Security solutions. In each episode, Microsoft Security evangelists Nic Fillingham and Natalia Godyla take a closer look at the latest innovations in threat intelligence, security research, and data science, with a special focus on demystifying artificial intelligence and machine learning. Be sure to listen in and subscribe!

See for privacy and opt-out information.

18 Episodes
Digital crime-fighter Donal Keating revisits the podcast, but this time… it’s personal. *cue dramatic crime-fighting music* The Director of Innovation and Research of the Digital Crimes Unit (DCU) at Microsoft joins hosts Nic Fillingham and Natalia Godyla to regale us with the origin story of the DCU and his captivating career exploits. Whether it’s tales of his early days preventing Windows 98 counterfeits in Ireland or the many international law enforcement raids he’s participated in…there’s no shortage to Donal’s crime-fighting adventures.In This Episode, You Will Learn:• The mission of Microsoft’s DCU and the techniques used to combat fraud• The events and needs that led to the creation of a forensic analytic lab at Microsoft• How counterfeiting and intellectual property crime have evolved over the years with advanced technology• What it’s like partnering with law enforcement to take down criminals around the worldSome Questions We Ask:• What does a day in the life of Donal look like in the DCU?• Was there ever a counterfeit example that shocked Donal at just how good it was?• With so many shifts in Donal’s work, what in his background has prepared him to stay on top of the changes?• What does a digital crime fighter do in their time off?Resources:  Donal’s LinkedIn  Nic’s LinkedIn      Natalia’s LinkedIn      Microsoft Security Blog   Transcript(Full transcript can be found at Fillingham:Hello, and welcome to Security Unlocked, a new podcast from Microsoft where we unlock insights from the latest in news and research from across Microsoft Security engineering and operations teams. I'm Nic Fillingham.Natalia Godyla:And I'm Natalia Godyla. In each episode, we'll discuss the latest stories for Microsoft Security, deep dive into the latest threat intel, research and data science-Nic Fillingham:And profile some of the fascinating people working on artificial intelligence in Microsoft Security.Natalia Godyla:And now, let's unlock the pod. Hi, Nic. How's it going?Nic Fillingham:Hello, Natalia. It's going well. How are you?Natalia Godyla:It's going well. I am super-excited for this episode, because it will be a trip down memory lane. We're gonna be talking about counterfeiting CDs and Beanie Babies. Well, Beanie Babies aren't covered in this episode, but they're counterfeited.Nic Fillingham:So we were, we were having a conversation before we started recording about, you know, things that have been counterfeited, and one of the examples that we stumbled upon was Beanie Babies, and I said, "What's a Beanie Baby?" And Natalia said "How do you not know what Beanie Babies are?" So 15 minutes ago, you, you educated me on a Beanie Baby, and I've learned something about you, is that you collected Beanie Babies. Is that right? You were in the Beanie Baby fad. You were in the, the trend.Natalia Godyla:Oh, yes. Yes. Beanie Babies and Pokemon cards. I definitely collected them.Nic Fillingham:Do you still have your Pokemon cards?Natalia Godyla:Yes. Yes, I do.Nic Fillingham:And do you still have your Beanie Babies?Natalia Godyla:I've got one Beanie Baby left.Nic Fillingham:Do you know, with certainty, that it is not a counterfeit Beanie Baby?Natalia Godyla:I don't, but I don't think I wanna find out.Nic Fillingham:If only there were some kind of technology. Maybe a, a hologram or something, embedded into the Beanie Baby for you to have a high degree of certainty-Natalia Godyla:(laughs)Nic Fillingham:That it was real.Natalia Godyla:(laughs)Nic Fillingham:And I'm talking about holograms because our guest on the podcast today, Donal Keating from the DCU, walks us through his journey into security, and his path to Microsoft, and how he spent a lot of his career in the anti-counterfeiting space. And we talked about CDs, we talked about counterfeiting CDs and optical discs. This was very exciting for me. We talk about the period in time when I was actually joining Microsoft, which was when Windows XP was coming out, and so the whole, you know, hologram on the CD, and you hold it up to the light, and there'd be different colors and pictures, like...Nic Fillingham:That was all very exciting. I guess that must have been early 2000s. That was a-, that was super-exciting when that was happening, so this was a, this was a great conversation, and I think we also talk about chickens at some point, too. I don't, I d-, I'm not sure how we got there, but we cover a lot of ground in this conversation.Natalia Godyla:And with that, I feel like we shouldn't keep people hanging. On to the pod.Nic Fillingham:On to the pod.Natalia Godyla:Hi, Donal. Welcome back to Security Unlocked. Thanks for joining us for a second time.Donal Keating:Thank you. I'm delighted to be here.Natalia Godyla:So Donal, you are the director of research and innovation of the Digital Crime Unit. I know that you've talked a little bit about what you did in our last episode, but would you mind giving the audience a refresher? What does a, a day in the life of Donal in the Digital Crimes Unit?Donal Keating:Well, e-each day is different, obviously, because when you're sort of working on the, on the side of security and crime-fighting, people evolve very rapidly, so there is no set pattern of what I do every day. But I am lucky to have a relatively unique position in the DCU, we call it the Digital Crimes Unit, in that I work across all of the different pillars th-, that we fight, and I also the opportunity to work, uh, work across the company, so... And we're always looking for new techniques, new data sources, and new crime mechanics, and I tend to get involved in, in the things that are new. So it's a very interesting job. As someone said, there's not many jobs where you wake up in the morning and look at the news and say, "What's going to be on my plate today?" But-Natalia Godyla:(laughs)Donal Keating:Working in this space tends to be that sort of a job.Natalia Godyla:And how did you end up in this role? What has been your path not just to Microsoft, but security? I know, a big question.Donal Keating:Oh, my. (laughs)Natalia Godyla:(laughs)Donal Keating:Once upon a time, Mammy Keating and Daddy Keating met. E-e-e-, um...Natalia Godyla:(laughs)Donal Keating:So, if I start-Nic Fillingham:And where was that, Donal?Natalia Godyla:(laughs)Donal Keating:Sorry? Wh- where was that?Nic Fillingham:Yeah, where was that?Donal Keating:That was in, that was in Ireland. So I, I grew up in-Nic Fillingham:Paint, paint us the picture. Like, it's, tell... I want beautiful, rolling green countrysides. I want-Donal Keating:(laughs)Nic Fillingham:Paint me that beautiful picture of Ireland.Donal Keating:Uh, well, uh... (laughs)Natalia Godyla:(laughs)Donal Keating:I don't know if I'm gonna go back that far. It's, that's before Moses was a boy.Nic Fillingham:(laughs)Natalia Godyla:(laughs)Donal Keating:So my parents are Irish. Uh, father an engineer, my grandfather an artist. My other grandfather was a blacksmith. So sort of technology had always been in the family. When I was growing up, uh, I guess my parents had been a product of the, of the war, and Ireland, at the best of times, didn't have very much, so the, the ability to make things and figure things out from first principles was always p-pretty important, uh, in my family.Donal Keating:So I grew up. My brother's, uh, an engineer. A c-, a civil engineer, built a very successful company in civil engineering. So I guess I was the black sheep of the family. I became a physicist, and when I graduated from physics, it was in the 1980s. I won't say exactly when, but the unemployment rate in Ireland at the time was in the high 20s, I believe, and for new graduates, there was pretty much two, three jobs a year going, and I certainly wasn't in the top two or three percent of the graduates coming out of the country, so I emigrated, like a lot of Irish people do, and my first stop was the UK.Donal Keating:So I got a job as a young, very green physicist. The only advantage I have is I had done applied physics, so I was to run a lathe as well as do some calculations, and I started to work for a, a UK company that was a venture capital-funded start up, looking at some very interesting optical technology. So my major was in opto-electronics, and this company was involved in the research into storage media. And at the time, CD audio had been quite the technology. C-, recordable CD had not been yet invented, but there was a space in the market for what was considered archival media, and this company had some very innovative and patented technology which we called Mothi. It was a, a recordable media that effectively made a mechanical mark. So it wasn't just a change of reflection. There was actually a mechanical mark on the media. And b- (laughs), I won't even go into the capacities of these things in, in today's world.Nic Fillingham:Almost like a vinyl record?Donal Keating:Uh, a-, well, uh, almost like a vinyl record, but at a nano scale. So a laser would... What normally it would do with m-, recordable media is, a dye would absorb, or not absorb and a-allow light through to the reflective layer beneath. The trick of this technology, called WORM, uh, write once read many, was a layer that looked a little bit like an egg box, and when the laser hit the texture, it would blow a bubble in the egg box, therefore making it reflective, and the company name was Plasmon, which actually refers to a physical phenomenon that means a surface that the, uh, incident light gets redirected along the surface of the incident plane.Donal Keating:So i-, it was just an interesting piece of technology. I worked for that company for six years, starting out knowing nothing, and worked for an incredible mentor engineer, a guy by the name of Bob Longman, who taught many engineers like me. He was quite a legend.Donal Keating:And through that company, it was like pure R&D work. We knew what the end goal was, but how to get there was entirely uncharted.So we got to work on all sorts of interesting, uh, technologies. But that really was the beginning of a skillset that I think everyone in security needs, and, uh, particular in research innovation. It's, when there aren't train tracks, how can you look at a problem, split it into smaller problems, and do things that you can measure, observe... Uh, basically articulate, "Well, okay, these three things happen. Therefore, what does it mean for the bigger picture?" So that reframing the question was training that I got right when I, when I graduated. So that was the start.Nic Fillingham:I think I i-interrupted you, Donal, but what was the... Did you tell us? What was the capacity? What was the storage capacity of this early CD-Donal Keating:(laughs) -Nic Fillingham:Technology. Nic Fillingham:I'm assuming it was small.Donal Keating:it was-Nic Fillingham:I'm assuming that's, that's Nic Fillingham:... the giggle your-Donal Keating:It was small, yeah.Natalia Godyla:(laughs)Donal Keating:540 megabytes was considered this huge enormous storage capacity.Nic Fillingham:But that's much smaller than the, the theoretical max of uh, cd's. No, it didn't say you only get to about 714 meg or something?Donal Keating:Yeah. Yeah but that, that was yeah but that was a CDR, and now we got DVDR, and yeah but these are capacities like if you pick a USB now, the tiny, tiny, tiny surface area will contain ten times that capacity. You know you look at floppy discs and you know, you look at the evolution of it. Really truly the laws of physics are being uh, like hard disc drives which I, at one stage I worked for Seagate, I'm like come to the, my narrative, but even when I was at Seagate in the 90's, the idea that you were coming close to the capacity of what a platter could hold.Donal Keating:They continue, hard drives, continue to push the limits. They're still uh, following Moore's at a phenomenal rate. Like if you look at a technology like hard drive, and you had to start that from scratch, people would say that's impossible. That is absolutely impossible to get that performance you know, even if using a huge design team.Donal Keating:But that's the great thing about evolution, you start off with something small you tweak it, you tweak it, you tweak it you put economic pressure on it to make it faster and bigger and you end up with here we have hard drives today same with Solid State. Solid State technology in another 20 years time. There will still be Solid State and it'll be faster and bigger and better than all the rest of it.Nic Fillingham:I thought you were sort of going to be comparing that early technology. That mech, that mechanical I forget the, the words you use but that mechanical mark on the disc. I thought you might have been comparing that to sort of later uh, technologies for writing to a CD. But you were, you were talking about CD's in general. Yes the capacity of a CD is, is obviously very very small.Donal Keating:Yeah. So the, the sort of people that were interested in it were people who needed archival technology. So uh, they worked with the British Library for instance was one of their um, audiences. But also company records and you know things that needed very good archival life. So, what you might not know is that your CDR um, if you've kept them in a drawer for 20 years will not be producing all the pictures that you thought you'd put onto your CDR.Donal Keating:And those technologies break down relatively quickly. So this was a, a technology that they said would um, stay on the shelf for a long time.Nic Fillingham:Why was that? The material is sort of susceptible to pressure change, temperature change, what, what is it?Donal Keating:Well with a recordable CD for instance is a dyeing. And dyes tend not to be, not to be stabled. You know you look at an old book even when it's closed up. The pictures in the, in your old books would be faded from what they were. Well if you need that high contrast and, and you have fading with your dye, you're gonna loose fidelity.Donal Keating:That's really just comparing this technology and CDR which is you know, but, the bit that I'm getting to is, you might have recording mechanisms that store data for a long time but the drives that read those do not store for a long time. Donal Keating:So, back then it was all scuzzy interfaces. To find a PC with a scuzzy interface now would be a, would be a whole, a whole piece of work. So, the reason the Cloud is gonna be so much better for storing data is regardless of what the readout technology is going to evolve with the Cloud.Donal Keating:I was kinda lucky in my career in that I was at the right place at the right time. So I worked for a number of companies that basically built CD manufacturing in Ireland. I hopped around those companies being part of the supply chain to Microsoft. So the very first indication of security, Microsoft introduced what we called an Innerband Hologram on I want to say was Windows 98.Donal Keating:It was a security feature to try and make counterfeiting of the Windows 98 dix, more difficult. Long story short, Microsoft decided themselves that they wanted a CD manufacturing plant. And they recruited me. At the time I really want to work for Microsoft. I had been a supplier to them and they had been pretty aggressive as customers. So I, I wasn't a terribly keen employee but they made it worth my while to join Microsoft to build them a CD-ROM plant in Dublin which I did.Donal Keating:We got that up and running. And just at that time, a team in the US wanted even more secure CD manufacturing. So at the time, one of the great ways of making money very easily was to produce either Office 97 or Windows 98 CD's and sell them. Now, you could make money in different ways. You could just bootleg them and make recordable CD's, but people then knew that they were buying something cheap and cheerful. There was, you get a few bucks for it but you weren't gonna make big dollars. Donal Keating:But the more sophisticated criminals did is they made visual pass offs, like very very good pass offs of the product. Packaged them up and even it into the supply chain. So today everyone is conscious of supply chain attacks. Solar winds being an example and in the recent past supply chain attacks have been all over the business. But if you go back to those times, people didn't really consider the supply chain attack. And one of the significant vulnerabilities in the software industry back then was, there was this whole world of people prepared to make very, very sophisticated counterfeits. Donal Keating:So, I was working for Microsoft at the time and there had been some legal cases chasing down counterfeiters and the, they had a newly appointed attorney in Europe looking after the counterfeiting team and we got talking and it was just one of those things that you know, you suddenly meet someone who knows what they really want to do and I knew how the product was made. And I said, "Look. All, all of the, the way you're going about this identification of counterfeit is all wrong." You know. The, the example I think was that if something was misprinted, it was, if it was badly printed disc it must be counterfeit. Donal Keating:I've run en, enough CD plants to know you can have a bad day in printing discs. So that was the start of the concept of a proper forensic analytic lab that would look at product and say, "This is genuine or counterfeit." And that really was the start of getting into the security space. And then I guess was in the year 2000-2001 maybe.Natalia Godyla:What was your next step within Microsoft. What, what brought you to the role you have today?Donal Keating:Yeah, so actually at the time when, when I met the legal team for the first time I, I was transitioning out from running the CD plant to working on the anti-counterfeiting technologies. In fact I used to, I kinda had a role that was mostly based in the US uh, looking at hologram technology, fingerprinting technology, just a variety of technologies that are going to be used to protect our products. Donal Keating:But it became more and more interesting to me to chase the criminals rather than to try and protect the product. There was lots of people focused on protecting the product. There was very few people uh, focused on, on locking up the crooks. And I think that was from one side, from the traditional counterfeiting side. One of the things that you got to learn is the economics of being a, a criminal.Donal Keating:And they would save themselves as, as people but what's their motivation? How do they do it? You know, how do they communicate? So, that was way back then that seemed to be very interesting and exciting. So I did more and more of that. Like I said I went around the world. I was in raids all over the world of, of plants producing counterfeit discs.Nic Fillingham:Can you share any examples?Natalia Godyla:(laughs)Donal Keating:Yeah. yeah, yeah, yeah I can so, the, the more recent one actually that's back in 2013 because we pretty much stopped em' physically counterfeiting but back in 2013-2014, there was a plant in the Ukraine that had been, it, it had belonged to the old regime. There's a new regime comes in so they re-raid the plant and I, I got called in just because I knew about how to obtain evidence from a CD plant. So they just wanted a kind of an expert from Microsoft to help them obtain Donal Keating:... obtain the evidence from the plant. But I arrived at this factory, brought there by law enforcement, and they had these huge doors, big, enormous, big steel doors. But the bit that appealed to me was (laughing) two feet to the right of the door, there was actually a hole blown in the wall. The cops said that to do the raid, he said, "That door is too secure but the wall's not so secure." So they went through the wall.Donal Keating:I- I've done cases in- in Russia also. So everyone knows that counterfeiting is a problem, but one of the ways you- you protect yourself is if you have someone who is on the law enforcement side of the house who will not raid plants, that they are kind of under their protection. But what happens when you stop paying the protection money? So it turns out that Microsoft got pulled in because someone wasn't paying their protection money, uh, anymore, and law enforcement raided the f- facility. Donal Keating:I went there to analyze the evidence and testify that yes, this in fact was a Microsoft product that was being counterfeited. When the plant that had been raided realized that the law enforcement were taking it seriously, they obviously paid their dues again. So I'm in this police station in the morning, uh, we're taking the evidence, y- you know, gathering up the notes. And when you're handling evidence, you have these tags, so you take something out, do your analysis, and then you seal the bag and- and sign it. Donal Keating:Suddenly, there's an urgent request to go to lunch at, you know, 11:30 or something. Never a man to dodge lunch, we went off to lunch.Natalia Godyla:(laughs).Donal Keating:But the lunch went on about three hours, and when we came back I'm looking at my pile and I see all this stuff that I had already examined, but they're not my seals, it's not my signature. And I said, "Th- this is not what I looked at this morning." (laughs). "Oh yeah, that's- that's what you looked at this morning." (laughs).Donal Keating:It was the sort of environment where you don't- don't go and argue with anyone, so we just stepped away from that. There was some- some follow-up, but there was no confirmation that what that plant had been producing was Microsoft counterfeits and it all got swept under the carpet.Nic Fillingham:Donal, when I hear the word raid though, I think of paramilitary, I think of guns and- and- and all that. Is my mental image accurate? What, how- how sort of scary, how dangerous were these- these raids that you were a part of? Or are they a bit more sort of... Well, yeah, that- that- that's my question.Donal Keating:So generally with counterfeiting, they tend to be, they're not dangerous. So sometimes, mostly I would get called in after the raid had happened, so therefore there's no danger, the environment is secure. Remember, these manufacturers are doing it on behalf of someone else. It- it's like malware today, there's a whole bunch of different individuals in the supply chain. My specialization at the time was the- the actual plants themselves, so we were going to sites that it was a regular manufacturer who was just breaking the law. There wasn't that risk.Donal Keating:But since I came to the US, I moved for Microsoft to the US in 2013, I got hauled into a raid where someone was selling product keys, and for some reason the case was a Homeland Security case. And that's the first time that I've ever seen, I actually wrote up a report afterwards, um, I was there with a- a Microsoft colleague and he was ex-FBI, and to him, it was perfectly normal. But to an Irishman who has grown up on American TV, it looked like the real thing. Donal Keating:They had an address and we were going in to the address, but there's a briefing beforehand that has a SWAT and a whole bunch of agents that are going there now. We're invited along as the- the analysts, like to- to analyze what they find. But there's this briefing that starts off with, you know, if there's- if there's shootings, here's where the hospitals are. If it's, you know, serious, here's where the helicopters land. You kinda get this mental image built up that you're going to raid a super-secure and heavily-armed target.Donal Keating:In this case, the entire team arrive up (laughing), and the guy arrives out in his dressing gown. And- and his first words to law enforcement was, "I haven't counterfeited for a year." (laughs). Natalia Godyla:(laughs).Nic Fillingham:(laughs). Donal Keating:Working that closely with law enforcement was quite a buzz, but all of that was sort of intellectual property crime that I was focused on, and since then, since 2013, 2014, I have changed my focus pretty much entirely to protecting Microsoft customers. So taking all of those techniques and, you know, understanding about the way people behave, and looking at behavior of criminals. Donal Keating:And using data, in essence, to- to look for, I used to look forensically for evidence of did it come from an authorized supply chain or an unauthorized supply chain? We built some special technology to do that with microscopes and image matching and stuff. So taking a lot of those concepts and then applying it to data streams. Is this a normal behavior for this type of data? Where's the anomaly? What's the cause of it? All of those sorts of things.Natalia Godyla:Was there ever a counterfeit example that shocked you, that was just so close to truth that you were surprised? Like just awed at the counterfeit artistry?Donal Keating:Well, I will say absolutely, I'm- I'm in awe of the ability for people to make things that look so visually identical. And a- a counterfeit never, they never manufacture things in exactly the same way that we did it, so we would emboss a hologram, uh, the counterfeiters by and large produce labels. But boy, were those labels good visual pass-offs. You know, it became, I wouldn't say impossible but it actually became, you know, you need to put your glasses on to look at the th- thing and say, "Oh yeah, that's counterfeit."Donal Keating:But again, that's to someone who has knowledge of the product. Uh, I think a- a thing that a lot of people forget, specialists, people who look at this stuff all the time will look at it and say, "Oh, well, that's, you know, it's missing the T and I've got a small I here. And look, this- this color is a bit off." To someone who buys this product once every three years or once every two years there's no build-up of a reference library of, "You know what? If it looks good, it must be genuine. And in fact, there's a little sticker on it that says this is genuine." (laughs). Therefore, you're socially engineered into thinking yes, it's genuine.Donal Keating:Uh, I love- I love when you g- get products from Amazon and you, a little card comes out that says, "This is an authentic product because, you know, we've got the card that says it's an authentic product."Nic Fillingham:The certificate of authenticity, which is a little matchbox-Donal Keating:Uh, yeah.Nic Fillingham:... square of cardboard that, uh, (laughing)-Donal Keating:Yeah, yeah. Nic Fillingham:... has been printed on an inkjet printer. (laughing). And cut out with scissors.Donal Keating:Yeah. One of the things that criminals are very good at is social engineering people into thinking they're doing the right thing, in- in whatever area it is. Like they would give people additional stuff in counterfeit packages, and made them feel even better about themselves getting this really good deal online. Uh, you know, it- it's just the- the psychology of- of people, we're just not designed to be suspicious of everything. Which is great, but unfortunately for people who work in this space, you get suspicious of everything.Nic Fillingham:So we're rapidly moving away from physical media. My Xbox doesn't even have a- doesn't even have a disc drive anymore, so it's, you know, it's entirely- it's entirely online digital distribution. But I see there is still, there are still counterfeiters out there. There are still, you know, it's still probably big business in some parts of the world. Is that, are- are, do you still have your finger on the pulse or have you fully, uh, left that- that space?Donal Keating:I have fully left that space, but absolutely, you know, there- as long as there is a dollar to be made there will be people in that space. But it's just not- it's just not what Microsoft focuses our effort on. You know, there- there will always be people who wanna go and pick up- up Windows on a CD-R. What I would say is then they know the risks that they're taking. You know, they're- they're a self-selecting group. Donal Keating:You know, we always talk about make sure that you're patched and have everything updated and use good password security. Well, you can- you can lose all that if you choose to obtain your software on a recordable CD where it says, you know, "This- this is real stuff." You know, e- especially on a, at the OS level. When you're installing an OS from a disc before anything has been turned on and all your signatures have been updated, it- it- Donal Keating:It's really easy to- to build a device with a lot of malware on it. Therefore, that is an area that I have concerns about, is that your supply chain for your hardware is, y- you're not buying the thing that you can get for the cheapest price. You're- you're buying from your authorized channel, you're buying from people that are reputable. Donal Keating:I think one of the really important things in security is the reputation and, you know, trustworthiness of your supply chain. So that's not an area that we spend a huge amount of time in, but it certainly is a thing that, um, is- is of concern to me.Nic Fillingham:And Donal, I think you've already said this, but to reiterate, the- the- the principles and the learning from your time in- in forensics and in physical, uh, disk manufacturing and- and- and anti-counterfeit work is that the sort of human psychology and the social engineering that was a big part of that business continues to this day. And you were sort of bringing a lot of those learnings and principles forward, and you're just now applying them to, uh, new supply chains and- and new technologies. Is that- is that accurate?Donal Keating:That's accurate. The- the one other thing, we did start to get into what I would consider big data in 2013, 2014, when we started to take activation behavior. So as devices touch Microsoft's servers for activation or validation, starting to do analysis on- on, at a large scale. So there were a lot of indications back when that you could identify countries that had relatively high rates of what I would considered piracy, and they correlated well with what, with encounter rates of malware coming from Defender and th- the various AV companies.Donal Keating:So it- it started out as a narrative, uh, in 2013, 2014, that we had high piracy rates. You als- also had high levels of- of security issues on the devices. I think that has- that has continued to some extent, but now as we move to a more digital, and- and hopefully more secure, supply chain, that opportunity for people to, you know, put large volumes of physical product that have malicious doors on them is hopefully being removed. But the skillset that I learned in, you know, analyzing very large volumes of data, that sort of was the start of it.Donal Keating:In fact, the Digital Crimes Unit built some analytic environments, uh, originally on, you know, on-prem servers, and now we've moved over to Azure. That allows us to do very large-scale analytics of huge datasets. That was sort of borne of our analysis of activation and validation, um, six, seven years ago.Natalia Godyla:You've had a couple notable shifts. What else other than your background in analytics has prepared you, or have you done to prepare for these changes? Do you have any recommendations to somebody who might be experiencing a similar shift and wants to get up to speed for this type of role?Donal Keating:W- well, if it's in Microsoft, we are incredibly lucky in that we have some very, very smart people. I'd say that the number one skillset that you need in navigating this is your ability to pick up the phone and talk to someone and admit that you know nothing about it. You really do have to talk to people who have expert knowledge in the area. Because you can be great at cultivating data, but unless you understand really what it means down to a very, very granular level, not the- not the 101 version of it but the 201 and 301 version of what do these things mean? And in Microsoft, we also have the people from Microsoft Research. I've been helped enormously on the AI and ML side from people who have done this clustering on short strings. Donal Keating:There is no magic to any of this. You've gotta have the data, you've gotta have the right data, you've gotta have the cleaned data, but there are tooling that, once you have everything that you want, allow you to represent it in a way that is easy to manipulate and- and highlight the things that are important. So I would say what have I done? I've talked to a lot of people in Microsoft about how they do what they're specialized at.Nic Fillingham:And what about when you're not working on this stuff? What's, what do you- what do you like to do, Donal, in your- in your spare time? And does any of that, uh, bleed over into your professional life? Do you, uh, do you like to do your thinking when you're climbing walls or- or something? That was a terrible example, but- but what (laughing)- what-Natalia Godyla:(laughs).Nic Fillingham:What do you- what do you do for fun?Donal Keating:Well, when I'm working, my 150-pound dog, who really is a- a- a slobbering sweetheart-Natalia Godyla:(laughs). Nic Fillingham:Type of dog, breed?Donal Keating:He's an Anatolian Shepherd, specifically a Kangal. So-Nic Fillingham:I have a Great Pyrenees, which I believe is a- a distant cousin. Donal Keating:Oh, yes. Yes. Uh, his name is Pamuk, it's a Turkish breed and pamuk means cotton in Turkish. But when I'm working, um, he does kinda, because he's a big dog, I kinda like to think that, you know, hey, if we had a security team that just looked, you know, dangerous would people mess with our product? Natalia Godyla:(laughs).Donal Keating:So that's one thing that, you know, I- I- I do like to think about my job when I walk the dog. But I'm also something of an urban farmer. I have three chickens and I like to grow potatoes, because I'm an Irishman, and turnips and leeks and stuff in my tiny little garden. So.Nic Fillingham:Are your chickens laying at the moment? Because we have ducks, and my ducks have gone on strike and I'm not getting any eggs out of them at the moment.Natalia Godyla:(laughs).Nic Fillingham:I'm wondering if- if you're... I know- I know chickens and ducks are a- a different bird, so I am aware of that, but just wondering if it's, what are you seeing in your- in your chickens?Donal Keating:You know, I'm a data guy, so, um, we went from one egg per chicken per day in the summer to kind of nothing in the late fall, and then starting luckily on the 21st of December, we got a- a burst of eggs. And then we now, out of three chickens I get one a day. I'm not exactly sure which one is doing, is... Natalia Godyla:(laughs).Donal Keating:If one is producing all of 'em or they're firing every third day. But, um, yeah, we're- we're- we're production again. Nic Fillingham:I think we need some machine learning algorithms to, uh, monitor the egg producing habits of chickens and/or ducks to see if we can, uh, increase output. Donal Keating:Uh, for- for sure.Natalia Godyla:(laughs). Donal Keating:It- it's- it's the only way to go about it, eh? The problem though with AI is we'd need to get about half a million chickens, and then we'd have a pretty good answer.Natalia Godyla:(laughs).Nic Fillingham:(laughs). Natalia Godyla:Well, we definitely thank you for that, Donal. And thanks for joining us again on Security Unlocked. Donal Keating:You're very welcome. Thank you for having me back. Natalia Godyla:Well, we had a great time unlocking insights into security. From research to artificial intelligence, keep an eye out for our next episode.Nic Fillingham:And don't forget to tweet us @msftsecurity, or email us at, with topics you'd like to hear on a future episode. Until then, stay safe.Natalia Godyla:Stay secure. See for privacy and opt-out information.
Most people know the age-old adage, “Don’t judge a book by its cover.” I can still see my grandmother wagging her finger at me when I was younger as she said it. But what if it's not the book cover we’re judging, but the title? And what if it’s not a book we’re analyzing, but instead a security bug? The times have changed, and age-old adages don’t always translate well in the digital landscape. In this case, we’re using machine learning (ML) to identify and “judge” security bugs based solely on their titles.  And, believe it or not, it works! (Sorry, Grandma!) Mayana Pereira, Data Scientist at Microsoft, joins hosts Nic Fillingham and Natalia Godyla to dig into the endeavors that are saving security experts’ time. Mayana explains how data science and security teams have come together to explore ways that ML can help software developers identify and classify security bugs more efficiently. A task that, without machine learning, has traditionally provided false positives or led developers to overlook misclassified critical security vulnerabilities.  In This Episode, You Will Learn:• How data science and ML can improve security protocols and identify and classify bugs for software developers • How to determine the appropriate amount of data needed to create an accurate ML training model • The techniques used to classify bugs based simply on their title  Some Questions We Ask:• What questions need to be asked in order to obtain the right data to train a security model? • How does Microsoft utilize the outputs of these data-driven security models?  • What is AI for Good and how is it using AI to foster positive change in protecting children, data and privacy online?  Resources:  Microsoft Digital Defense Report  Article: “Identifying Security Bug Reports Based Solely on Report Titles and Noisy Data”  Mayana’s LinkedIn  Nic’s LinkedIn        Natalia’s LinkedIn        Microsoft Security Blog: Transcript(Full transcript can be found at Fillingham:Hello, and welcome to Security Unlocked, a new podcast from Microsoft where we unlock insights from the latest in news and research from across Microsoft Security engineering and operations teams. I'm Nic Fillingham-Natalia Godyla:And I'm Natalia Godyla. In each episode we'll discuss the latest stories from Microsoft Security, deep dive into the newest threat, intel, research and data science-Nic Fillingham:And profile some of the fascinating people working on artificial intelligence in Microsoft Security.Natalia Godyla:And now let's unlock the pod.Natalia Godyla:Hello, Nic. How's it going?Nic Fillingham:Hello, Natalia. Welcome back. Well, I guess welcome back to Boston to you. But welcome to Episode 16. I'm confused because I saw you in person last week for the first time. Well, technically it was the first time for you, 'cause you didn't remember our first time. It was the second time for me. But it was-Natalia Godyla:I feel like I just need to justify myself a little bit there. It was a 10 second exchange, so I feel like it's fair that I, I was new to Microsoft. There was a lot coming at me, so, uh-Nic Fillingham:Uh, I'm not very memorable, too, so that's the other, that's the other part, which is fine. But yeah. You were, you were here in Seattle. We both did COVID tests because we filmed... Can I say? You, you tell us. What did we do? It's a secret. It is announced? What's the deal?Natalia Godyla:All right. Well, it, it's sort of a secret, but everyone who's listening to our podcast gets to be in the know. So in, in March you and I will be launching a new series, and it's a, a video series in which we talk to industry experts. But really we're, we're hanging with the industry experts. So they get to tell us a ton of really cool things about [Sec Ups 00:01:42] and AppSec while we all play games together. So lots of puzzling. Really, we're just, we're just getting paid to do puzzles with people cooler than us.Nic Fillingham:Speaking of hanging out with cool people, on the podcast today we have Mayana Pereira whose name you may have heard from a few episodes ago Scott Christiansen was on talking about the work that he does. And he had partnered Mayana to build and launch a, uh, machine learning model that looked at the titles of bugs across Microsoft's various code repositories, and using machine learning determined whether those bugs were actually security related or not, and if they were, what the correct severity rating should be. Nic Fillingham:So this episode we thought we'd experiment with the format. And instead of having two guests, instead of having a, a deep dive upfront and then a, a profile on someone in the back off, we thought we would just have one guest. We'd give them a little bit extra time, uh, about 30 minutes and allow them to sort of really unpack the particular problem or, or challenge that they're working on. So, yeah. We, we hope you like this experiment.Natalia Godyla:And as always, we are open to feedback on the new format, so tweet us, uh, @msftsecurity or send us an email Let us know what you wanna hear more of, whether you like hearing just one guest. We are super open. And with that, on with the pod?Nic Fillingham:On with the pod.Nic Fillingham:Welcome to the Security Unlocked podcast. Mayana Pereira, thanks for joining us.Mayana Pereira:Thank you for having me. I'm so happy to be here today, and I'm very excited to share some of the things that I have done in the intersection of [ML 00:03:27] and security.Nic Fillingham:Wonderful. Well, listeners of the podcast will have heard your name back in Episode 13 when we talked to Scott Christiansen, and he talked about, um, a fascinating project about looking for or, uh, utilizing machine learning to classify bugs based simply on, on their title, and we'll get to that in a minute. But could you please introduce you- yourself to our audience. Tell us about your title, but sort of what does that look like in terms of day-to-day and, and, and the work that you do for Microsoft?Mayana Pereira:I'm a data scientist at Microsoft. I've been, I have been working at Microsoft for two years and a half now. And I've always worked inside Microsoft with machine learning applied to security, trust, safety, and I also do some work in the data privacy world. And this area of ML applications to the security world has always been my passion, so before Microsoft I was also working with ML applied to cyber security more in the malware world, but still security. And since I joined Microsoft, I've been working on data science projects that kinda look like this project that we're gonna, um, talk today about. So those are machine learning applications to interesting problems where we can either increase the trust and the security Microsoft products, or the safety for the customer. You know, you would develop m- machine learning models with that in mind. Mayana Pereira:And my day-to-day work includes trying to understand which are those interesting programs across the company, talk to my amazing colleagues such as Scott. And I have a, I have been so blessed with an amazing great team around me. And thinking about these problems, gathering data, and then getting, you know, heads down and training models, and testing new machine learning techniques that have never been used for a specific applications, and trying to understand how well or if they will work for those applications, or if they're gonna get us to better performance, or better accuracy precision and those, those metrics that we tend to use in data science works. And when we feel like, oh, this is an interesting project and I think it is interesting enough to share with the community, we write a paper, we write a blog, we go to a conference such as RSA and we present it to the community, and we get to share the work and the findings with colleagues internal to Microsoft, but also external. So this is kinda what I do on a day-to-day basis.Mayana Pereira:Right now my team is the data science team inside Microsoft that is called AI For Good, so the AI for Good has this for good in a sense of we want to, to guarantee safety, not only for Microsoft customers, but for the community in general. So one of my line of work is thinking about how can I collaborate with NGOs that are also thinking about the security or, and the safety of kids, for example. And this is another thing that I have been doing as part of this AI for Good effort inside Microsoft.Natalia Godyla:Before we dive into the bug report classification project, can you just share a couple of the projects that your team works for AI for Good? I think it would be really interesting for the audience to hear that.Mayana Pereira:Oh, absolutely. So we have various pillars inside the AI for Good team. There is AI for Health, AI for Humanitarian Action, AI for Earth. We have also been collaborating in an effort for having a platform with a library for data privacy. It is a library where we have, uh, various tools to apply the data and get us an output, data with strong privacy guarantees. So guaranteeing privacy for whoever was, had their information in a specific dataset or contributed with their own information to a specific research and et cetera. So this is another thing that our team is currently doing.Mayana Pereira:And we have various partners inside and outside of Microsoft. Like I mentioned, we do a lot of work in NGOs. So you can think like project like AI for Earth several NGOs that are taking care of endangered species and other satellite images for understanding problems with the first station and et cetera. And then Humanitarian Action, I have worked with NGOs that are developing tools to combat child sexual abuse and exploration. AI for Health has so many interesting projects, and it is a big variety of projects. Mayana Pereira:So this is what the AI for Good team does. We are, I think right now we're over 15 data scientists. All of us are doing this work that it is a- applied research. Somehow it is work that we need to sit down with, with our customers or partners, and really understand where the problem is. It's usually some, some problems that required us to dig a little deeper and come up with some novel or creative solution for that. So this is basically the overall, the AI for Good team.Nic Fillingham:Let's get back in the way back machine to I think it was April of 2020, which feels like 700 years ago.Mayana Pereira:(laughs) Nic Fillingham:But you and Scott (laughs) published a blog. Scott talked about on Episode 13 called securing Nic Fillingham:The s- the software development lifecycle with machine learning, and the thing that I think both Natalia and I picked up on when Scott was talking about this, is it sounded first-, firstly it sounded like a exceptionally complex premise, and I don't mean to diminish, but I think Natalia and I were both "oh wow you built a model that sort of went through repro steps and passed all the logs inside of security bugs in order to better classify them but that's not what this does", this is about literally looking at the words that are the title of the security bug, and then building a model to try and determine whether it was truly security or something else, is that right?Mayana Pereira:That's exactly it. This was such an interesting project. When I started collaborating with Scott, and some other engineers in the team. I was a little skeptical about using only titles, to make prediction about whether a bug has, is security related or not. And, it seems. Now that I have trained several models and passed it and later retrained to- to get more of a variety of data in our model. I have learned that people are really good at describing what is going on in a bug, in the title, it feels like they really summarize it somehow so it's- it's doing a good job because, yes, that's exactly what we're doing, we are using bug titles only from several sources across Microsoft, and then we use that to understand which bugs are security related or not, and how we can have an overall view of everything that is happening, you know in various teams across different products. And, that has given a lot of visibilities to some unknown problems and some visibility to some things that we were not seeing before, because now you can scan, millions of bugs in a few seconds. Just reading titles, you have a model that does it really fast. And, I think it is a game changer in that sense, in the visibility and how do you see everything that is happening in that bug world.Natalia Godyla:So what drove that decision? Why are we relying only on the titles, why can't we use the- the full bug reports? Mayana Pereira:There are so many reasons for that. I think, the first reason was the fact that the full bug report, sometimes, has sensitive information. And we were a little bit scared about pulling all that sensitive information which could include passwords, could include, you know, maybe things that should not be available to anyone, and include that in a- in a VM to train a model, or, in a data science pipeline. And, having to be extremely careful also about not having our model learning passwords, not having that. So that was one of the big, I think incentives off, let's try titles only, and see if it works. If it doesn't work then we can move on and see how we can overcome the problem of the sensitive information. And it did work, when we saw that we had a lot of signal in bug titles only, we decided to really invest in that and get really good models by u- utilizing bug titles only. Nic Fillingham:I'm going to read from the blog just for a second here, because some of the numbers here, uh, are pretty staggering, so, again this was written 2020, uh, in April, so there's obviously, probably updated numbers since then but it said that Microsoft 47,000 developers generate nearly 30,000 bugs a month, which is amazing that's coming across over 100 Azure DevOps and GitHub repositories. And then you had it you, you actually have a count here saying since 2001 Microsoft has collected 13 million work items and bugs which I just thinks amazing. So, do you want to speak to, sort of, the volume of inputs and, sort of, signals here in to building that model and maybe some of the challenges, and then a follow on question is, is this model, still active today, is this- is this work still ongoing, has it been incorporated into a product or another, another process?Nic Fillingham:Do you want to start with, with numbers or. Mayana Pereira:Yes, I think that from my data scientist point of view, having such large numbers is absolutely fantastic because it gives us a historical data set, very rich so we can understand how data has evolved over time. And also, if this- the security terminology has changed the law, or how long will this model last, in a sense. And it was interesting to see that you can have different tools, different products, different things coming up, but the security problems, at least for, I would say for the past six, seven years, when it comes to terminology, because what I was analyzing was the terminology of the security problems. My model was a natural language processing model. It was pretty consistent, so that was really interesting to see from that perspective we have. And by having so much data, you know, this amazing volume. It helped us to build better classifiers for sure. So this is my- my data scientist side saying, amazing. I love it so much data.Nic Fillingham:What's the status of this project on this model now.? Is it- is it still going? Has it been embedded into another- another product, uh, or process?Mayana Pereira:Yes, it's still active. It's still being used. So, right now, this product. This, not the product- the product, but the model is mainly used by the customer security interest team in [Sila 00:16:16], so they use the model in order to understand the security state of Microsoft products in general, and, uh, different products and looking at specific products as well, are using the model to get the- the bugs statistics and security bugs statistics for all these different products across Microsoft. And there are plans on integrating the- this specific model or a variation of the model into other security lifecycle pipelines, but this is a decision that is more on CST customer Security Trust side and I have, um, only followed it, but I don't have specific details for that right now. But, I have seen a lot of good interesting results coming out of that model, good insights and security engineers using the results of the model to identify potential problems, and fix those problems much faster.Natalia Godyla:So, taking a step back and just thinking about the journey that your team has gone on to get the model to the state that it's in today. Uh, in the blog you listed a number of questions to figure out what would be the right data to train the model. So the questions were, is there enough data? How good is the data? Are there data usage restrictions? And, can data be generated in a lab? Natalia Godyla:So can you talk us through how you answered these questions like, as a- as a data scientist you were thrilled that there was a ton of data out there, but what was enough data? How did you define how good the data was? Or, whether it was good enough.Mayana Pereira:Great. So, those were questions that I asked myself before even knowing what the project was about, and the answer to is there enough data? It seemed very clear from the beginning that, yes, we had enough data, but those were questions that I brought up on the blog, not only for myself but for anyone else that was interested in replicating those experiments in their company or maybe university or s- anywhere any- any data scientist that is interested to train your own model for classification, which questions should be asked? Once you start a project like this. So the, is there enough data for me? Was clear from the beginning, we had several products so we had a variety of data sources. I think that when you reach, the number of millions of samples of data. I think that speaks for itself. It is a high volume. So I felt, we did have enough data.Mayana Pereira:And, when it came to data quality. That was a more complex question. We had data in our hands, bugs. We wanted to be able to train a model that could different- differentiate from security bugs and non security bugs, you know. And, for that, Usually what we do with machine learning, is we have data, that data has labels, so you have data that represents security bugs, data that represents non security bugs. And then we use that to train the model. And those labels were not so great. So we needed to understand how the not so great labels was going to impact our model, you know, we're going to train a model with labels that were not so great. So Mayana Pereira:That was gonna happen. So that was one of the questions that we asked ourselves. And I did a study on that, on understanding what is the impact of these noisy labels and the training data set. And how is it gonna impact the classification results that we get once using this, this training data? So this was one of the questions that I asked and we, I did several experiments, adding noise. I did that myself, I, I added noise on purpose to the data set to see what was the limits of this noise resilience. You know, when you have noisy labels in training, we published it in a, in an academic conference in 2019, and we understood that it was okay to have noisy labels. So security bugs that were actually labeled as not security and not security bugs labeled as security. There was a limit to that.Mayana Pereira:We kinda understood the limitations of the model. And then we started investigating our own data to see, is our own data within those limits. If yes, then we can use this data confidentially to train our models. If no, then we'll have to have some processes for correcting labels and understanding these data set a little bit better. What can we use and what can we not use to train the models. So what we found out is that, we didn't have noisy labels in the data set. And we had to make a few corrections in our labels, but it was much less work because we understood exactly what needed to be done, and not correct every single data sample or every single label in a, an enormous data set of millions of entries. So that was something that really helped. Mayana Pereira:And then the other question, um, that we asked is, can we generate data in the lab? So we could sometimes force a specific security issue and generate some, some box that had that security description into titles. And why did we include that in the list of questions? Because a lot of bugs that we have in our database are generated by automated tools. So when you have a new tool being included in your ecosystem, how is your model going to recognize the bugs that are coming from this new tool? So does our, ma- automatically generated box. And we could wait for the tool to be used, and then after a while we gathered the data that the tool provided us and including a retraining set. But we can also do that in the lab ecosystem, generate data and then incorporate in a training set. So this is where this comes from.Nic Fillingham:I wanted to ask possibly a, a very rudimentary question, uh, especially to those that are, you know, very familiar with machine learning. When you have a data set, there's words, there is text in which you're trying to generate labels for that text. Does the text itself help the process of creating labels? So for example, if I've got a bug and the name of that bug is the word security is in the, the actual bug name. Am I jump-starting, am I, am I skipping some steps to be able to generate good labels for that data? Because I already have the word I'm looking for. Like I, I think my question here is, was it helpful to generate your labels because you were looking at text in the actual title of the bug and trying to ascertain whether something was security or not?Mayana Pereira:So the labels were never generated by us or by me, the data scientists. The labels were coming from the engineering systems where we collected the data from. So we were relying on what- whatever happened in the, in the engineering team, engineering group and relying that they did, uh, a good job of manually labeling the bugs as security or not security. But that's not always the case, and that doesn't mean that the, the engineers are not good or are bad, but sometimes they have their own ways of identifying it in their systems. And not necessarily, it is the same database that we had access to. So sometimes the data is completely unlabeled, the data that comes to us, and sometimes there are mistakes. Sometimes you have, um, specific engineer that doesn't have a lot of security background. The person sees a, a problem, describes the problem, but doesn't necessarily attribute the problem as a security problem. Well, that can happen as well.Mayana Pereira:So that is where the labels came from. The interesting thing about the terminology is that, out of the millions and millions of security bugs that I did review, like manually reviewed, because I kinda wanted to understand what was going on in the data. I would say that for sure, less than 1%, even less than that, had the word security in it. So it is a very specific terminology when you see that. So people tend to be very literal in what the problem is, but not what the problem will generate. In a sense of they will, they will use things like Cross-site Scripting or passwords in clear, but not necessarily, there's a security pr- there's a security problem. But just what the issue is, so it is more of getting them all to understand that security lingual and what is that vocabulary that constitutes security problems. So that's wh- that's why it is a little bit hard to generate a list of words and see if it matches. If a specific title matches to this list of words, then it's security.Mayana Pereira:It was a little bit hard to do that way. And sometimes you have in the title, a few different words that in a specific order, it is a security problem. In another order, it is not. And then, I don't have that example here with me, but I, I could see some of those examples in the data. For example, I think the Cross-site Scripting is a good example. Sometimes you have site and cross in another place in the title. It has nothing to do with Cross-site Scripting. Both those two words are there. The model can actually understand the order and how close they are in the bug title, and make the decision if it is security or not security. So that's why the model is quite easier to distinguish than if we had to use rules to do that.Natalia Godyla:I have literally so many questions. Nic Fillingham:[laughs].Natalia Godyla:I'm gonna start with, uh, how did you teach at the lingo? So what did you feed the model so that it started to pick up on different types of attacks like Cross-site Scripting?Mayana Pereira:Perfect. The training algorithm will do that for me. So basically what I need to guarantee is that we're using the correct technique to do that. So the technique will, the machine learning technique will basically identify from this data set. So I have a big data set of titles. And each title will have a label which is security or non-security related to it. Once we feed the training algorithm with all this text and their associated labels, the training algorithm will, will start understanding that, some words are associated with security, some words are associated with non-security. And then the algorithm will, itself will learn those patterns. And then we're gonna train this algorithm. So in the future, we'll just give the algorithm a new title and say, "Hey, you've learned all these different words, because I gave you this data set from the past. Now tell me if this new ti- if this new title that someone just came up with is a security problem or a, a non-security problem." And the algorithm will, based on all of these examples that it has seen before, will make a decision if it is security or non-security.Natalia Godyla:Awesome. That makes sense. So nothing was provided beforehand, it was all a process of leveraging the labels. Mayana Pereira:Yes.Natalia Godyla:Also then thinking about just the dataset that you received, you were working with how many different business groups to get this data? I mean, it, it must've been from several different product teams, right?Mayana Pereira:Right. So I had the huge advantage of having an amazing team that is a data center team that is just focused on doing that. So their business is go around the company, gather data and have everything harmonized in a database. So basically, what I had to do is work with this specific team that had already done this amazing job, going across the company, collecting data and doing this hard work of harvesting data and harmonizing data. And they had it with them. So it is a team that does that inside Microsoft. Collects the data, gets everything together. They have their databases updated several times a day, um, collecting Mayana Pereira:... Data from across the company, so it is a lot of work, yeah.Natalia Godyla:So do different teams treat bug reports differently, meaning is there any standardization that you had to do or anything that you wanted to implement within the bug reports in order to get better data?Mayana Pereira:Yes. Teams across the company will report bugs differently using different systems. Sometimes it's Azure DevOps, sometimes it can be GitHub. And as I mentioned, there is a, there was a lot of work done in the data harmonization side before I touched the data. So there was a lot of things done to get the data in, in shape. This was something that, fortunately, several amazing engineers did before I touched the data. Basically, what I had to do once I touched it, was I just applied the data as is to the model and the data was very well treated before I touched it. Nic Fillingham:Wow. So many questions. I did wanna ask about measuring the success of this technique. Were you able to apply a metric, a score to the ... And I'm, I, I don't even know what it would be. Perhaps it would be the time to address a security bug pre and post this work. So, did this measurably decrease the amount of time for prioritized security bugs to be, to be addressed?Mayana Pereira:Oh, definitely. Yes, it did. So not only it helped in that sense, but it helped in understanding how some teams were not identifying specific classes of bugs as security. Because we would see this inconsistency with the labels that they were including in their own databases. These labels would come to this big database that is harmonized and then we would apply the model on top of these data and see that specific teams were treating their, some data points as non-security and should have been security. Or sometimes they were treating as security, but not with the correct severity. So it would, should have been a critical bug and they were actually treating it as a moderate bug. So, that, I think, not only the, the timing issue was really important, but now you have a visibility of behavior and patterns across the company that the model gives us.Nic Fillingham:That's amazing. And so, so if I'm an engineer at Microsoft right now and I'm in my, my DevOps environment and I'm logging a bug and I use the words cross- cross scripting somewhere in the bug, what's the timing with which I get the feedback from your model that says, "Hey, your prioritization's wrong," or, "Hey, this has been classified incorrectly"? Are we at the point now where this model is actually sort of integrated into the DevOps cycle or is that still coming further down the, the, the path?Mayana Pereira:So you have, the main customer is Customer Security and Trust team inside Microsoft. They are the ones using it. But as soon as they start seeing problems in the data or specific patterns and problems in specific teams' datasets, they will go to that team and then have this, they have a campaign where they go to different teams and, and talk to them. And some teams, they do have access to the datasets after they are classified by our model. Right now, there's, they don't have the instant response, but that's, that's definitely coming.Nic Fillingham:So, Mayana, how is Customer Security and Trust, your organization, utilizing the outputs of this model when a, when a, when a bug gets flagged as being incorrectly classified, you know, is there a threshold, and then sort of what happens when you, when you get those flags?Mayana Pereira:So the engineering team, the security engineering team in Customer Security and Trust, they will use the model to understand the overall state of security of Microsoft products, you know, like the products across the company, our products, basically. And they will have an understanding of how fast those bugs are being mitigated. They'll have an understanding of the volume of bugs, and security bugs in this case, and they can follow this bugs in, in a, in a timely manner. You know, as soon as the bug comes to the CST system, they bug gets flagged as either security or not security. Once it's flagged as security, there, there is a second model that will classify the severity of the bug and the CST will track these bugs and understand how fast the teams are closing those bugs and how well they're dealing with the security bugs.Natalia Godyla:So as someone who works in the AI for Good group within Microsoft, what is your personal passion? What would you like to apply AI to if it, if it's not this project or, uh, maybe not a project within Microsoft, what is, what is something you want to tackle in your life?Mayana Pereira:Oh, love the question. I think my big passion right now is developing machine learning models for eradication of child sexual abuse medias in, across different platforms. So you can think about platform online from search engines to data sharing platforms, social media, anything that you can have the user uploading content. You can have problems in that area. And anything where you have using visualizing content. You want to protect that customer, that user, from that as well. But most importantly, protect the victims from those crimes and I think that has been, um, something that I have been dedicating s- some time now. I was fortunate to work with an NGO, um, recently in that se- in that area, in that specific area. Um, developed a few models for them. She would attacked those kind of medias. And these would be my AI for Good passion for now. The other thing that I am really passionate about is privacy, data privacy. I feel like we have so much data out there and there's so much of our information out there and I feel like the great things that we get from having data and having machine learning we should not, not have those great things because of privacy compromises. Mayana Pereira:So how can we guarantee that no one's gonna have their privacy compromised? And at the same time, we're gonna have all these amazing systems working. You know, how can we learn from data without learning from specific individuals or without learning anything private from a specific person, but still learn from a population, still learn from data. That is another big passion of mine that I have been fortunate enough to work in such kind of initiatives inside Microsoft. I absolutely love it. When, when I think about guaranteeing privacy of our customers or our partners or anyone, I think that is also a big thing for me. And that, that falls under the AI for Good umbrella as well since that there's so much, you know, personal information in some of these AI for Good projects. Natalia Godyla:Thank you, Mayana, for joining us on the show today.Nic Fillingham:We'd love to have you back especially, uh, folks, uh, on your team to talk more about some of those AI for Good projects. Just, finally, where can we go to follow your work? Do you have a blog, do you have Twitter, do you have LinkedIn, do you have GitHub? Where should, where should folks go to find you on the interwebs?Mayana Pereira:LinkedIn is where I usually post my latest works, and links, and interesting things that are happening in the security, safety, privacy world. I love to, you know, share on LinkedIn. So m- I'm Mayana Pereira on LinkedIn and if anyone finds me there, feel free to connect. I love to connect with people on LinkedIn and just chat and meet new people networking.Natalia Godyla:Awesome. Thank you. Mayana Pereira:Thank you. I had so much fun. It was such a huge pleasure to talk to you guys.Natalia Godyla:Well, we had a great time unlocking insights into security from research to artificial intelligence. Keep an eye out for our next episode. Nic Fillingham:And don't forget to Tweet us at MSFTSecurity or email us at with topics you'd like to hear on a future episode. Until then, stay safe. Natalia Godyla:Stay secure.  See for privacy and opt-out information.
Prior to the pandemic, workdays used to look a whole lot different. If you had a break, you could take a walk to stretch your legs, shake the hands of your co-workers, or get some 1-on-1 face time with the boss. Ahh... those were the days. That close contact we once had is now something that many of us yearn for as we’ve had to abruptly lift and shift from living in our office to working from our home. But communicating and socializing aren’t the only things that were easier back then. The walls of your office have expanded, and with them, the boundaries of your security protocols. Small in-office tasks like patching a server have now become multi-step processes that require remote management, remote updates, and remote administrative control. With that comes the prioritization of resilience and what it means for enterprises, customers, and security teams alike. That’s where remote enterprise resiliency comes into play.  Today on the pod, we explore the final chapter of the MDDR. Irfan Mirza, Director of Enterprise Continuity and Resilience at Microsoft, wraps up the observations from the report by giving hosts Nic Fillingham and Natalya Godyla the rundown on enterprise resiliency and discusses how we can ensure the highest levels of security while working from home. Irfan explains the Zero trust model and how Microsoft is working to extend security benefits to your kitchen or home office, or...  that make-shift workspace in your closet.  In the second segment, Andrew Paverd, Senior Researcher on the Microsoft Security Response Center Team and jack of all trades, stops by… and we’re not convinced he’s fully human. He’s here to tell us about the many hats he wears, from safe systems programming to leveraging AI to help with processes within the MSRC, and shares how he has to think like a hacker to prevent attacks. Spoiler alert: he’s a big follower of Murphy’s Law.   In This Episode, You Will Learn:  • How classical security models are being challenged • What the Zero Trust Model is and how it works  • The three critical areas of resilience: extending the enterprise boundary, prioritizing resilient performance, and validating the resilience of our human infrastructure.  • How hackers approach our systems and technologies  Some Questions We Ask: • How has security changed as a product of the pandemic? • Do we feel like we have secured the remote workforce? • What frameworks exist to put a metric around where an organization is in terms of its resiliency? • What is Control Flow Guard (CFG) and Control-Flow Integrity? • What’s the next stage for the Rust programming language?  Resources: Microsoft Digital Defense Report: Irfan’s LinkedIn Andrew’s LinkedIn Nic’s LinkedIn    Natalia’s LinkedIn    Microsoft Security Blog:   Transcript(Full transcript can be found at Fillingham:Hello, and welcome to Security Unlocked, a new podcast from Microsoft, where we unlock insights from the latest in news and research from across Microsoft security, engineering and operations teams. I'm Nic Fillingham.Natalia Godyla:And I'm Natalia Godyla. In each episode, we'll discuss the latest stories from Microsoft Security, deep dive into the newest threat intel, research and data science. Nic Fillingham:And profile some of the fascinating people working on artificial intelligence in Microsoft Security. Natalia Godyla:And now let's unlock the pod. Hi Nic, I have big news.Nic Fillingham:Big news. Tell me a big news.Natalia Godyla:I got a cat. Last night at 8:00 PM, I got a cat. Nic Fillingham:Did it come via Amazon Prime drone? Natalia Godyla:No.Nic Fillingham:Just, that was a very specific time. Like 8:00 PM last night is not usually the time I would associate people getting cats. Tell me how you got your cat. Natalia Godyla:It was a lot more conventional. So I had an appointment at the shelter and found a picture of this cat with really nubby legs and immediately-Nic Fillingham:(laughs).Natalia Godyla:... fell in love obviously. And they actually responded to us and we went and saw the cat, got the cat. The cat is now ours. Nic Fillingham:That's awesome. Is the cat's name nubby. Natalia Godyla:It's not, but it is on the list of potential name changes. So right now the cat's name is tipper. We're definitely nervous about why the cat was named tipper. Nic Fillingham:(laughs).Natalia Godyla:We're hiding all of the glass things for right now. Nic Fillingham:How do we get to see the cat? Is there, will there be Instagram? Will there be Twitter photos? This is the most important question.Natalia Godyla:Wow. I haven't planned that yet.Nic Fillingham:You think about that and I'll, uh, I'll start announcing the first guest on this episode.Natalia Godyla:(laughs).Nic Fillingham:On today's episode, we speak with Irfan Mirza, who is wrapping up our coverage of the Microsoft Digital Defense Report with a conversation about enterprise resiliency. Now, this is really all of the chapters that are in the MDDR, the nation state actors, the increase in cyber crime sophistication, business email compromise that you've heard us talk about on the podcast, all gets sort of wrapped up in a nice little bow in this conversation where we talk about all right, what does it mean, what does it mean for customers? What does it mean for enterprises? What does it mean for security teams? And so we talk about enterprise resiliency. And we actually recorded this interview in late 2020, but here we are, you know, two months later and those findings are just as relevant, just as important. It's a great conversation. And after that, we speak with-Natalia Godyla:Andrew Paverd. So he is a senior researcher on the Microsoft Security Response Center team. And his work is well, well, he does a ton of things. I honestly don't know how he has time to pull all of this off. So he does everything from safe systems programming to leveraging AI, to help with processes within MSRC, the Microsoft Security Response Center. And I just recall one of the quotes that he said from our conversation was hackers don't respect your assumptions, or something to that effect, but it's such a succinct way of describing how hackers approach our systems and technology. So another really great conversation with a, a super intelligent researcher here at Microsoft.Nic Fillingham:On with the pod.Natalia Godyla:On with the pod. Today, we're joined by Irfan Mirza, Director of Enterprise Continuity and Resilience, and we'll be discussing the Microsoft Digital Defense Report and more specifically enterprise resilience. So thank you for being on the show today, Irfan.Irfan Mirza:Thanks so much glad to be here. And hope we have a, a great discussion about this. This is such an important topic now. Natalia Godyla:Yes, absolutely. And we have been incrementally working through the Microsoft Digital Defense Report, both Nic and I have read it and have had some fantastic conversations with experts. So really looking forward to hearing about the summation around resilience and how that theme is pulled together throughout the report. So let's start it off by just hearing a little bit more about yourself. So can you tell us about your day-to-day? What is your role at Microsoft? Irfan Mirza:Well, I lead the enterprise continuity and resilience team and we kind of provide governance overall at the enterprise. We orchestrate sort of all of the, the risk mitigations. We go and uncover what the gaps are, in our enterprise resilience story, we try to measure the effectiveness of what we're doing. We focus on preparedness, meaning that the company's ready and, you know, our critical processes and services are always on the ready. It's a broad space because it spans a very, very large global enterprise. And it's a very deep space because we have to be experts in so many areas. So it's a fun space by saying that.Natalia Godyla:Great. And it's really appropriate today then we're talking about the MDDR and enterprise resilience. So let's start at a high level. So can you talk a little bit about just how security has changed as a product of the pandemic? Why is resilience so important now? Irfan Mirza:Yeah, it's a great question. A lot of customers are asking that, our field is asking that question, people within the company are asking. Look, we've been 11 months under this pandemic. Maybe, you know, in some places like China, they've been going through it for a little bit longer than us, you know, a couple of months more. What we're finding after having sort of tried to stay resilient through this pandemic, uh, one obviously is on the human side, everyone's doing as much as we possibly can there. But the other part of it is on the enterprise side. What is it that we're having to think about as we think of security and as we think of enterprise resilience?Irfan Mirza:There are a couple of big things that I think I would note, one is that, look, when this pandemic hit us, our workforce lifted and shifted. I mean, by that, I mean that we, we, we got up out of our offices and we all left. I mean, we took our laptops and whatever we could home. And we started working remotely. It was a massive, massive lift and shift of personnel, right? We got dispersed. Everybody went to their own homes and most of us have not been back to the office. And it's not just at Microsoft, even, even a lot of our customers and our partners have not gone back to the office at all, right? So that, that's a prolong snow day, if you want to call it that.Irfan Mirza:The other thing that happened is our workload went with us. Wasn't just that, "Hey, you know, I'm taking a few days off, I'm going away or going on vacation and, and I'll be checking email periodically." No, I actually took our work with us and we started doing it remotely. So what that's done is it's created sort of a, a need to go back and look at what we thought was our corporate security boundary or perimeter.Irfan Mirza:You know, in the classical model, we used to think of the corporation and its facilities as the, the area that we had to go and secure. But now in this dispersed workforce model, we have to think about my kitchen as part of that corporate perimeter. And all of a sudden we have to ensure that, that my kitchen is as secure as the corporate network or as the facilities or the office that I was working from. That paradigm is completely different than anything we'd thought about before. Nic Fillingham:And so Irfan, in the MDDR, uh, this section, um, and if you've got the report open, you're playing along at home, I believe it's page 71. This enterprise resiliency is sort of a wrap-up of, of a lot of the observations that are in the MDDR report. It's not a new section. It's as you're getting towards the end of the report, you're looking for, okay, now what does this mean to me? I'm a CSO. I need to make new security policies, security decisions for my organization. This concept of enterprise resiliency is sort of a wrap up of everything that we've seen across cyber crime, across the nation state, et cetera, et cetera. Is that, is that accurate? Is that a good way to sort of read that section in the report? Irfan Mirza:Yeah. It is really the, the way to think of it, right.? It's sort of like a, the conclusion, so what, or why is this relevant to me and what can I do about it? When you think about the report and the way that it's structured, look, we, you know, the report goes into great detail about cyber crime as you called out Nic. And then it talks about nation state threats.Irfan Mirza:These are newer things to us. We've certainly seen them on the rise, actors that are well-trained, they're well-funded they play a long game, not necessarily a short game, they're looking, they're watching and they're waiting, they're waiting for us to make mistakes or to have gaps, they look for changes in tactics, either ours, uh, they themselves are quite agile, right? Irfan Mirza:So when you think about the environment in which we have to think about resilience, and we have to think about security, that environment itself has got new vectors or new threats that are, that are impacting it, right? In addition to that, our workforce has now dispersed, right? We're all over the, all over the globe. We see emerging threats that are, that are, non-classical like ransomware. We see attacks on supply chain. We continue to see malware and malware growing, right? Irfan Mirza:And, and so when you think about that, you have to think if I need to secure now my, my dispersed corporate assets and resources, my people, the workload, the data, the services and the processes that are all there, what are the, the sort of three big things I would need to think about? And so this report sort of encapsulates all, all of that. It gives the details of what, what's happening. And, and then page 71 is you say that resilience piece sort of comes back and says, "Look, your security boundaries extended. Like it or not, it is extended at this point. You've got to think beyond that on-site perimeter that we were thinking about before."Irfan Mirza:So we have to start thinking differently. And th- there's three critical areas that are sort of called out, acknowledging the security boundary has increased, thinking about resilience and performance, and then validating the resilience of our human infrastructure. This is like new ideas, but these are all becoming imperatives for us. We're having to do this now, whether we like it or not. Irfan Mirza:And so this report sort of gives our customers, and, and it's a reflection of what we're doing in the company. It's an open and honest conversation about how we propose to tackle these challenges that we're facing.Nic Fillingham:And so Irfan if we can move on to that critical area, number two, that prioritizing resilient performance. When I say the word performance and resilient performance, is that scoped down just to sort of IT infrastructure, or does that go all the way through to the humans, the actual people in the organization and, um, how they are performing their own tasks, their own jobs and the tasks that are part of their, their job and et cetera, et cetera? What's the, I guess what's the scope of that area too?Irfan Mirza:As we were thinking about resilience, as you know, shortly after we dispersed the workforce, we started thinking about, about what should be included in our classical understanding of resilience. But when you think about, about typical IT services and online services, and so on, a lot of that work is already being done with the life site reviews that we do and people are paying very close attention to service performance. We have SLAs, we have obligations, we have commitments that we've made that our services will be performing to a certain degree, but there are also business processes that are associated with these services very closely. Irfan Mirza:When you think about all of the processes that are involved and services that are involved from the time a customer thinks of buying Office, uh, 365, as an example, to the time that they provision their first mailbox, or they receive their first email, there are dozens of process, business processes. Irfan Mirza:Every single service in that chain could be working to 100% efficiency. And yet if the business processes, aren't there, for instance, to process the deal, to process the contract, to process, uh, the customer's payment or, uh, acknowledge receipt of the payment in order to be able to provision the service, all of these processes, all of a sudden have to, we have to make sure that they're also performing.Irfan Mirza:So when we start thinking about resilience, up to now, business continuity has focused on, are you ready? Are you prepared? Are your dependencies mapped? Have you, have you done a business impact analysis? Are you validating and testing your preparedness? You know, are you calling down your call tree for instance? But I think where we're going now with true enterprise resilience, especially in this sort of modern Irfan Mirza:... day, we're, we're looking at performance, right? What, what is your preparedness resulting in? So if you stop and you think about a child at school, they get homework. Well, the homework really, they bring it home. They do it. They take it back to the teacher. They get graded on it. That's wonderful. This means that the child is ready. But at some point in time, the class or the teacher is going to give them a test, and that test is going to be the measure of performance, right? Irfan Mirza:So we need to start thinking of resilience and continuity in the same way. We're prepared. We've done all our homework. Now let's go and see how many outages did you have? How critical were the outages? How long did they last? How many of them were repeat outages? How many of the repeat outages were for services that are supposed to have zero downtown, like services that are always supposed to on like your DNS service or your identity auth- authentication service, right? So, when you start thinking about, uh, resilience from that perspective, now you've got a new set of data that you have to go and capture, or data that you're capturing, you have to now have to have insights from it. You've got to be able to correlate your preparedness, meaning the homework that you've done with your actual performance, your outage and your, and your gap information. All right?Irfan Mirza:So that, that's what prioritizing resilient performance is all about. It's about taking realtime enterprise preparedness and mapping it to real time enterprise performance. That tells you if your preparedness is good enough or not, or what it is that you need to do. There's a loop here, a feedback loop that has to be closed. You can't just say that, well, you know, we've done all the exercises theoretically. We're good and we're ready to take on any sort of a crisis or, or, or disaster. Yeah, that's fine. Can we compare it to realtime what you're doing? Can we break glass and see what that looks like? Can we shut you down and or shut down parts of your operation as in the event of an earthquake for instance, or a hurricane wiping out, uh, access to a data center, right? Can we do those things and still be resilient when that happens? So this is what performance and resilience come together in that space.Natalia Godyla:So am I right in understanding that beyond, like you said, the theoretical where you think about the policies that you should have in place, and the frameworks that you should have in place, you have the analytics on, you know, the state of, the state of how performant your systems are to date. And then in addition, is there now the need for some sort of stress testing? Like actually figuring out whether an additional load on a system would cause it to break, to not be resilient? Is that now part of the new approach to resilience?Irfan Mirza:Yeah. There are, there are several, several things to do here, right? You absolutely said it. There's a stress test. Actually, this pandemic has, is already a stress test in and of itself, right? It's stressing us in a many ways. It's stressing, obviously the psyche and, and, you know, our whole psychology, and our ability to sustain in quarantine, in isolated, in insulated environments and so on. But it's also testing our ability to do the things that we just so, uh, so much took for granted, like the ability to patch a server that's sitting under my desk in the office whenever I needed to, right? That server now has to become a managed item that somebody can manage remotely, patch remotely, update remotely when needed, control administrative access and privileges remotely. But yes, for resilience, I think we need to now collect all of the data that we have been collecting or looking at and saying, can we start to create those correlations between our preparedness and between our real performance? Irfan Mirza:But there's another area that this dovetails into which is that of human resilience, right? We talked a little bit earlier about, you know, sort of the whole world enduring this hardship. We need to first and foremost look at our suppliers, subcontractors, people that we're critically dependent on. What is their resilience look like? That's another aspect that we have to go back. In the areas where we have large human resources or, or workforces that are working on our behalf, we need to make sure that they're staying resilient, right? Irfan Mirza:We talked on a lot about work/life balance before. Now I think the new buzzword in HR conference rooms is going to be work/life integration. It's completely integrated, and so we need to start thinking about the impact that would have. Are we tracking attrition of our employees, of certain demographics within the employees? Are we looking at disengagement? People just sort of, "Yeah, I'm working from home, but I'm not really being fully engaged." Right? The hallway conversations we used to have are no longer there. And we need to start thinking, are people divesting? Our resources, are they divesting in the workplace? Are they divesting in their, in their work or work/life commitment? These measures are all now having to be sort of like... Irfan Mirza:We used to rely on intuition, a look, a hallway gaze, look at the, the snap in somebody's walk as they walked away from you or out of your office. We don't have that anymore. Everybody's relatively stagnant. We're, we're, we're seated. We don't get to see body language that much. We don't get to read that. There's a whole new set of dynamics that are coming into play, and I think smart corporations and smart companies will start looking at this as a very important area to pay attention to.Nic Fillingham:How are we measuring that? What tools or sort of techniques, or, or sort of frameworks exist to actually put a metric around this stuff, and determine sort of where, where an organization is in terms of their level of resiliency?Irfan Mirza:This question is actually the whole reason why we brought this enterprise resilience sort of a conclusion to this fourth chapter, and, and, you know, the summation of this, of this report. Irfan Mirza:What we're doing now is we're saying, look. Things that used to be fundamentally within the domain of IT departments, or used to be fundamentally with, within the domain of live site, or used to be fundamentally in the domain of human resource departments are now all floating up to be corporate imperatives, to be enterprise imperatives. I think the thinking here is that we need to make sure that the data that we've been collecting about, as an example to answer your question, attrition, right? A certain demographic. Millennials, uh, changing jobs, leaving the company, just to pick an example more than anything else. This is no longer just data that the HR Department is interested in, or that recruiting would be interested in, or, or retention would be interested. This is data that's about to significantly impact the enterprise, and it needs to be brought into the enterprise purview.Irfan Mirza:Our classical and traditional models of looking at things in silos don't allow us to do that. What we're recommending is that we need to have a broader perspective and try to drive insights from this that do tell a more comprehensive story about our ent- enterprise resilience. That story needs to include the resilience of our services, our business processes, our suppliers, our human capital, our infrastructure, our extended security boundary, our data protection, uh, prevention of data loss, our intrusion detection. I mean, there's such a broad area that we have to cover. That's we're saying. And, and as we implement this new sort of zero trust model, I think the, the effectiveness of that model, how much progress we're making is becoming an enterprise priority, not just something that the IT department is going to go around on it's own.Nic Fillingham:Irfan, I wonder if I could put you on the spot, and were there any interesting bits of data that you saw in those first couple months of the shift to remote work where like, yeah, the number of unique devices on the Microsoft corporate network quadrupled in 48 hours. Like any, anything like that? I'd just wondering what, what little stats you may have in hand.Irfan Mirza:Yeah. The number of devices and sort of the flavors of devices, we've always anticipated that that's going to be varied. We're cognizant of that. Look, we have, you know, people have PCs. They have MACs. They have Linux machines, and, and they have service o- operating software. There's a lot of different flavors. And, and it's not just the device and the OS that matters, it's also what applications you're running. Some applications we can certify or trust, and others perhaps we can't, or that we still haven't gotten around to, to verifying, right? And all of these sit, and they all perform various functions including intruding and potentially exfiltrating data and Spyware and Malware and all of that. So when you think about that, we've always anticipated it. Irfan Mirza:But the one thing that, that we were extremely worried about, and I think a lot of our Enterprise customers were worried about, is the performance of the workforce. What we found very early on in, in the, in the lift and shift phase was that we needed to have a way of measuring is our, our built processes working? Are we checking in the same amount of code as we were before? And we noted a couple of interesting things. We looked at our, our VPN usage and said, what are those numbers look like? Are they going up and down?Irfan Mirza:And I think what we found is that initially, the effect was quite comparable to what we had, uh, when we experienced snow days. Schools are shut down. People don't go to work. They're slipping and sliding over here. We're just not prepared for snow weather in, in this state like some of the others. So what happened is, we saw that we were, we were sort of seeing the same level of productivity as snow days. We say that we had the same level of VPN usage as snow days, and we were worried because that, you know, when, when it snows, people usually take the day off, and then they go skiing. Irfan Mirza:So what happened? Well, after about a week things started picking back up. People got tired of sort of playing snow day and decided that, you know what? It's time to, to dig in, and human nature, I think, kicked in, the integrity of the workforce kicked in. And sure enough, productivity went up, VPN usage went up, our number of sessions, the duration of sessions. Meetings became shorter.Nic Fillingham:Can I tell you hallelujah? (laughs) Irfan Mirza:(laughs) Nic Fillingham:That's one of the, that's one of the great-Irfan Mirza:Absolutely.Nic Fillingham:... upsides, isn't it? To this, this new culture of remote work is that we're all meeting for, for less amount of time, which I think, I think is fantastic.Irfan Mirza:Look, you know, in times of crisis, whether it's a natural disaster, or a pandemic, or, or a manmade situation such as a war or a civil war, or whatever, I, I think what happens is the amount of resources that you are customarily used to having access to gets limited. The way in which you work shifts. It changes. And so the, the true test of resilience, I think, is when you are able to adapt to those changes gracefully without requiring significant new investment and you're able to still meet and fulfill your customer obligations, your operational expectations. That really is.Irfan Mirza:So what you learn in times of hardship are to sort of live, you know, more spartan-like. And that spartan-ism, if there's such a word as that, that's what allows you to stay resilient, to say what are the core things that I need in order to stay up and running? And those fundamental areas become the areas of great investment, the areas that you watch over more carefully, the areas that you measure the performance of, the areas that you look for patterns and, and trends in to try to predict what's happening, right?Irfan Mirza:So that is something that carries over from experiences of being in the front lines of a, uh, a war or, or from being, uh, you know, in the midst of a hurricane trying to recover a data center, or an earthquake, or any other, uh, type of power outage, right? These are all the sort of key scenarios that we would be going to look at. And that's one of the things they all have in common. It's really that you don't have the resources or access to the resources that you thought you did, and now you've got to be able to do some things slightly differently.Natalia Godyla:Thank you for joining us on the podcast today. It's been great to get your perspective on enterprise resilience. Really fascinating stuff. So, thank you.Irfan Mirza:Thank you, Natalia. And, and thank you, Nick. It's been a great conversation. As I look back at this discussion that we had, I feel even, even stronger now that the recommendations that we're making, and the guidance that we're giving our customers and sharing our experiences, becomes really, really important. I think this is something that we're learning as we're going along. We're learning on the journey. We're uncovering things that we didn't know. We're looking at data in a different way. We're, we're trying to figure out how do we sustain ourselves, Nic Fillingham:... not just through this pandemic, but also beyond that. And I think the, whatever it is that we're learning, it becomes really important to share. And for our customers and people who are listening to this podcast to share back with us what they've learned, I think that becomes incredibly important because as much as we like to tell people what we're doing, we also want to know what, what people are doing. And so learning that I think will be a great, great experience for us to have as well. So thank you so much for enabling this conversation. Natalia Godyla:And now let's meet an expert from the Microsoft security team to learn more about the diverse backgrounds and experiences of the humans creating AI and tech at Microsoft. Welcome back to another episode of Security Unlocked. We are sitting with Andrew Paverd today, senior researcher at Microsoft. Welcome to the show, Andrew. Andrew Paverd:Thanks very much. And thanks for having me. Natalia Godyla:Oh, we're really excited to chat with you today. So I'm just doing a little research on your background and looks like you've had a really varied experience in terms of security domains consulting for mobile device security. I saw some research on system security. And it looks like now you're focused on confidential computing at Microsoft. So let's start there. Can you talk a little bit about what a day in the life of Andrew looks like at Microsoft? Andrew Paverd:Absolutely. I think I have one of the most fascinating roles at Microsoft. On a day-to-day basis, I'm a researcher in the confidential computing group at the Microsoft Research Lab in Cambridge, but I also work very closely with the Microsoft Security Response Center, the MSRC. And so these are the folks who, who are dealing with the frontline incidents and responding to reported vulnerabilities at Microsoft. But I work more on the research side of things. So how do we bridge the gap between research and what's really happening on the, on the front lines? And so I, I think my position is quite unique. It's, it's hard to describe in any other way than that, other than to say, I work on research problems that are relevant to Microsoft security. Natalia Godyla:And what are some of those research problems that you're focused on? Andrew Paverd:Oh, so it's actually been a really interesting journey since I joined Microsoft two years ago now. My background, as you mentioned, was actually more in systems security. So I had, I previously worked with technologies like trusted execution environments, but since joining Microsoft, I've worked on two really, really interesting projects. The, the first has been around what we call safe systems programming languages. Andrew Paverd:So to give a bit more detail about it in the security response center, we've looked at the different vulnerabilities that Microsoft has, has patched and addressed over the years and seen some really interesting statistics that something like 70% of those vulnerabilities for the pa- past decade have been caused by a class of vulnerability called memory corruption. And so the, the question around this is how do we try and solve the root cause of problem? How do we address, uh, memory corruption bugs in a durable way? Andrew Paverd:And so people have been looking at both within Microsoft and more broadly at how we could do this by transitioning to a, a different programming paradigm, a more secure programming language, perhaps. So if you think of a lot of software being written in C and C++ this is potentially a, a cause of, of memory corruption bugs. So we were looking at what can we do about changing to safer programming languages for, for systems software. So you might've heard about new languages that have emerged like the Rust programming language. Part of this project was investigating how far we can go with languages like Rust and, and what do we need to do to enable the use of Rust at Microsoft.Natalia Godyla:And what was your role with Rust? Is this just the language that you had determined was a safe buyable option, or were you part of potentially producing that language or evolving it to a place that could be safer? Andrew Paverd:That's an excellent question. So in, in fact it, it was a bit of both first determining is this a suitable language? Trying to define the evaluation criteria of how we would determine that. But then also once we'd found Rust to be a language that we decided we could potentially run with, there was an element of what do we need to do to bring this up to, let's say to be usable within Microsoft. And actually I, I did quite a bit of work on, on this. We realized that, uh, some Microsoft security technologies that are available in our Microsoft compilers weren't yet available in the Rust compiler. One in particular is, is called control flow guard. It's a Windows security technology and this wasn't available in Rust. Andrew Paverd:And so the team I, I work with looked at this and said, okay, we'd like to have this implemented, but nobody was available to implement it at the time. So I said, all right, let me do a prototype implementation and, uh, contributed this to the open source project. And in the end, I ended up following through with that. And so I've, I've been essentially maintaining the, the Microsoft control flow guide implementation for the, the Rust compiler. So really an example of Microsoft contributing to this open source language that, that we hope to be using further.Nic Fillingham:Andrew, could you speak a little bit more to control flow guard and control flow integrity? What is that? I know a little bit about it, but I'd love to, for our audience to sort of like expand upon that idea. Andrew Paverd:Absolutely. So this is actually an, an example of a technology that goes back to a collaboration between the MSRC, the, the security response center and, and Microsoft Research. This technology control flow guard is really intended to enforce a property that we call control flow integrity. And that simply means that if you think of a program, the control flow of a program jumps through two different functions. And ideally what you want in a well-behaved program is that the control always follows a well-defined paths. Andrew Paverd:So for example, you start executing a function at the beginning of the function, rather than halfway through. If for example, you could start executing a function halfway through this leads to all kinds of possible attacks. And so what control flow guard does is it checks whenever your, your program's going to do a bronch, whenever it's going to jump to a different place in the code, it checks that that jump is a valid call target, that you're actually jumping to the correct place. And this is not the attacker trying to compromise your program and launch one of many different types of attacks.Nic Fillingham:And so how do you do that? What's the process by which you do en- ensure that control flow?Andrew Paverd:Oh, this is really interesting. So this is a technology that's supported by Windows, at the moment it's only available on, on Microsoft Windows. And it works in conjunction between both the compiler and the operating system. So the compiler, when you compile your program gives you a list of the valid code targets. It says, "All right, here are the places in the program where you should be allowed to jump to." And then as the program gets loaded, the, the operating system loads, this list into a highly optimized form so that when the program is running it can do this check really, really quickly to say, is this jump that I'm about to do actually allowed? And so it's this combination of the Windows operating system, plus the compiler instrumentation that, that really make this possible. Andrew Paverd:Now this is quite widely used in Windows. Um, we want in fact as much Microsoft software as possible to use this. And so it's really critical that we enable it in any sort of programming language that we want to use. Nic Fillingham:How do you protect that list though? So now you, isn't that now a target for potential attackers?Andrew Paverd:Absolutely. Yeah. And, and it becomes a bit of a race to, to-Nic Fillingham:Cat and mouse.Andrew Paverd:... protect different-Natalia Godyla:(laughs).Andrew Paverd:A bit of, a bit of a cat, cat and mouse game. But at least the nice thing is because list is in one place, we can protect that area of memory to a much greater degree than, than the rest of the program. Natalia Godyla:So just taking a step back, can you talk a little bit about your path to security? What roles have you had? What brought you to security? What's informing your role today? Andrew Paverd:It's an interesting story of how I ended up working in security. It was when I was applying for PhD programs, I had written a PhD research proposal about a topic I thought was very interesting at the time on mobile cloud computing. And I still think that's a hugely interesting topic. And what happened was I sent this research proposal to an academic at the University of Oxford, where I, I was looking to study, and I didn't hear anything for, for a while. Andrew Paverd:And then, a fe- a few days later I got an email back from a completely different academic saying, "This is a very interesting topic. I have a project that's quite similar, but looking at this from a security perspective, would you be interested in doing a PhD in security on, on this topic?" And, so this was my very mind-blowing experience for me. I hadn't considered security in that way before, but I, I took a course on security and found that this was something I was, I was really interested in and ended up accepting the, the PhD offer and did a PhD in system security. And that's really how I got into security. And as they say, the rest is history.Natalia Godyla:Is there particular part of security, particular domain within security that is most near and dear to your heart?Andrew Paverd:Oh, that's a good question.Natalia Godyla:(laughs).Andrew Paverd:I think, I, I think for me, security it- itself is such a broad field that we need to ensure that we have security at, at all levels of the stack, at all, places within the chain, in that it's really going to be the weakest link that an attacker will, will go for. And so I've actually changed field perhaps three times so far. This is what keeps it interesting. My PhD work was around trusted computing. And then as I said, I, since joining Microsoft, I've been largely working in both safe systems programming languages and more recently AI and security. And so I think that's what makes security interesting. The, the fact that it's never the same thing two days in a row.Natalia Godyla:I think you hit on the secret phrase for this show. So AI and security. Can you talk a little bit about what you've been doing in AI and security within Microsoft? Andrew Paverd:Certainly. So about a year ago, as many people in the industry realized that AI is being very widely used and is having great results in so many different products and services, but that there is a risk that AI algorithms and systems themselves may be attacked. For example, I, I know you had some, some guests on your podcast previously, including Ram Shankar Siva Kumar who discussed the Adversarial ML Threat Matrix. And this is primarily the area that I've been working in for the past year. Looking at how AI systems can be, can be attacked from a security or a privacy perspective in collaboration with researchers, from MSR, Cambridge. Natalia Godyla:What are you most passionate about? What's next for a couple of these projects? Like with Rust, is there a desire to make that ubiquitously beyond Microsoft? What's the next stage? Andrew Paverd:Ab- absolutely. Natalia Godyla:Lots of questions. (laughs).Andrew Paverd:Yeah. There's a lot of interest in this. So, um, personally, I'm, I'm not working on the SSPL project myself, or I'm, I'm not working on the safe systems programming languages project myself any further, but I know that there's a lot of interest within Microsoft. And so hopefully we'll see some exciting things e- emerging in that space. But I think my focus is really going to be more on the, both the security of AI, and now we're also exploring different areas where we can use AI for security. This is in collaboration, more with the security response center. So looking into different ways that we can automate different processes and use AI for different types of, of analysis. So certainly a lot more to, to come in that space.Nic Fillingham:I wanted to come back to Rust for, for a second there, Andrew. So you talked about how the Rust programming language was specifically designed for, correct me on taxonomy, memory integrity. Is that correct?Andrew Paverd:For, for memory safety, yeah. Nic Fillingham:Memory safety. Got it. What's happening on sort of Nic Fillingham:... and sort of the, the flip side of that coin in terms of instead of having to choose a programming language that has memory safety as sort of a core tenet. What's happening with the operating system to ensure that languages that maybe don't have memory safety sort of front and center can be safer to use, and aren't threats or risks to memory integrity are, are sort of mitigated. So what's happening on the operating system side, is that what Control Flow Guard is designed to do? Or are there other things happening to ensure that memory safety is, is not just the responsibility of the programming language?Andrew Paverd:Oh, it's, that's an excellent question. So Control Flow Guard certainly helps. It helps to mitigate exploits once there's been an, an initial memory safety violation. But I think that there's a lot of interesting work going on both in the product space, and also in the research space about how do we minimize the amount of software that, that we have to trust. If you accept that software is going to have to bugs, it's going to have vulnerabilities. What we'd like to do, is we'd like to trust as little software as possible.Andrew Paverd:And so there's a really interesting effort which is now available in, in Azure under the, the heading of Confidential Computing. Which is this idea that you want to run your security sensitive workloads in a hardware enforced trusted execution environment. So you actually want to take the operating system completely out of what we call the trusted computing base. So that even if there are vulnerabilities in, in the OS, they don't affect your security sensitive workloads. So I think that there's this, this great trend towards confidential computing around compartmentalizing and segmenting the software systems that we're going to be running.Andrew Paverd:So removing the operating system from the trusted computing. And, and indeed taking this further, there's already something available in Azure, you can look up Azure Confidential Computing. But there's a lot of research coming in from the, the academic side of things about new technologies and new ways of, of enforcing separation and compartmentalization. And so I think it's part of this full story of, of security that we'll need memory safe programming languages. We'll need compartmentalization techniques, some of which, uh, rely on new hardware features. And we need to put all of this together to really build a, a secure ecosystem.Nic Fillingham:I only heard of Confidential Computing recently. I'm sure it's not a new concept. But for me as a sort of a productized thing, I only sort of recently stumbled upon it. I did not realize that there was this gap, there was this delta in terms of data being encrypted at rest, data being encrypted in transit. But then while the data itself was being processed or transformed, that that was a, was a gap. Is that the core idea around Confidential Computing to ensure that at no stage the data is not encrypted? Is, is that sort of what it is?Andrew Paverd:Absolutely. And it's one of the key pieces. So we call that isolated execution in the sense that the data is running in a, a trusted environment where only the code within that environment can access that data. So if you think about the hypervisor and the operation system, all of those can be outside of the trusted environment. We don't need to trust those for the correct computation of, of that data. And as soon as that data leaves this trusted environment, for example if it's written out of the CPU into the DRAM, then it gets automatically encrypted.Andrew Paverd:And so we have that really, really strong guarantee that only our code is gonna be touching our data. And the second part of this, and this is the really important part, is a, a protocol called remote attestation where this trusted environment can prove to a remote party, for example the, the customer, exactly what code is going to be running over that data. So you have a, a very high degree of assurance of, "This is exactly the code that's gonna be running over my data. And no other code will, will have access to it."Andrew Paverd:And the incredibly interesting thing is then, what can we build with these trusted execution environment? What can we build with Confidential Computing? And to bring this back to the, the keyword of your podcast, we're very much looking at confidential machine learning. How do we run machine learning and AI workloads within these trusted execution environments? And, and that unlocks a whole lot of new potential.Nic Fillingham:Andrew, do you have any advice for people that are m- maybe still studying or thinking about studying? Uh, I see so you, your initial degree was in, not in computer engineering, was it?Andrew Paverd:No. I, I actually did electrical engineering. And then electrical and computer engineering. And by the time I did a PhD, they put me in a computer science department, even though-Nic Fillingham:(laughs).Andrew Paverd:... I was doing software engineering.Nic Fillingham:Yeah. I, so I wonder if folks out there that, that don't have a software or a computer engineering degree, maybe they have a, a different engineering focus or a mathematics focus. Any advice on when and how to consider computer engineering, or sort of the computing field?Andrew Paverd:Yeah. Uh, absolutely. Uh, I think, eh, in particular if we're talking about security, I'd say have a look at security. It's often said that people who come with the best security mindsets haven't necessarily gone through the traditional programs. Uh, of course it's fantastic if you can do a, a computer science degree. But if you're coming at this from another area, another, another aspect, you bring a unique perspective to the world of cyber security. And so I would say, have a look at security. See if it's something that, that interests you. You, you might find like I did that it's a completely fascinating topic.Andrew Paverd:And the from there, it would just be a question of seeing where your skills and expertise could best fit in to the broad picture of security. We desperately need people working in this field from all different disciplines, bringing a diversity of thought to the field. And so I, I'd highly encourage people to have a look at this.Natalia Godyla:And you made a, quite a hard turn into security through the PhD suggestion. It, like you said, it was one course and then you were off. So, uh, what do you think from your background prepared you to make that kind of transition? And maybe there's something there that could inform others along the way.Andrew Paverd:I think, yes, it, it's a question of looking at, uh, of understanding the system in as much detail as you possibly can. And then trying to think like, like an attacker. Trying to think about what could go wrong in this system? And as we know, attackers won't respect our assumptions. They will use a system in a different way in which it was designed. And that ability to, to think out of the box, which, which comes from understanding how the system works. And then really just a, a curiosity about security. They call it the security mindset, of perhaps being a little bit cautious and cynical. To say-Natalia Godyla:(laughs).Andrew Paverd:... "Well, this can go wrong, so it probably will go wrong." But I think that's, that's the best way into it.Natalia Godyla:Must be a strong follower of Murphy's Law.Andrew Paverd:Oh, yes.Natalia Godyla:(laughs).Nic Fillingham:What are you watching? What are you binging? What are you reading? Either of those questions, or anything along in that flavor.Andrew Paverd:I'll, I'll have to admit, I'm a, I'm a big fan of Star Trek. So I've been watching the new Star Trek Discovery series on, on Netflix. That's, that's great fun. And I've recently been reading a, a really in- interesting book called Atomic Habits. About how we can make some small changes, and, uh, how these can, can help us to build larger habits and, and propagate through.Nic Fillingham:That's fascinating. So that's as in looking at trying to learn from how atoms and atomic models work, and seeing if we can apply that to like human behavior?Andrew Paverd:Uh, no. It's just the-Nic Fillingham:Oh, (laughs).Andrew Paverd:... title of the book.Natalia Godyla:(laughs).Nic Fillingham:You, you had me there. Natalia Godyla:Gotcha, Nick.Nic Fillingham:I was like, "Wow-"Natalia Godyla:(laughs).Nic Fillingham:" ... that sounds fascinating." Like, "Nope, nope. Just marketing." Marketing for the win. Have you always been Star Trek? Are you, if, if you had to choose team Star Trek or team Star Wars, or, or another? You, it would be Star Trek?Andrew Paverd:I think so. Yeah.Nic Fillingham:Yeah, me too. I'm, I'm team Star Trek. Which m- may lose us a lot of subscribers, including Natalia.Andrew Paverd:(laughs).Nic Fillingham:Natalia has her hands over her mouth here. And she's, "Oh my gosh." Favorite Star Trek show or-Andrew Paverd:I, I have to say, it, it would've been the first one I watched, Deep Space Nine.Nic Fillingham:I love Deep Space Nine. I whispered that. Maybe that-Natalia Godyla:(laughs).Nic Fillingham:... it's Deep Space Nine's great. Yep. All right, cool. All right, Andrew, you're allowed back on the podcast. That's good.Andrew Paverd:Thanks.Natalia Godyla:You're allowed back, but I-Nic Fillingham:(laughs).Natalia Godyla:... (laughs).Andrew Paverd:(laughs).Nic Fillingham:Sort of before we close, Andrew, is there anything you'd like to plug? I know you have a, you have a blog. I know you work on a lot of other sorta projects and groups. Anything you'd like to, uh, plug to the listeners?Andrew Paverd:Absolutely, yeah. Um, we are actually hiring. Eh, well, the team I work with in Cambridge is, is hiring. So if you're interested in privacy preserving machine learning, please do have a look at the website, And submit an application to, to join our team.Natalia Godyla:That sounds fascinating. Thank you.Nic Fillingham:And can we follow along on your journey and all the great things you're working at, at your website?Andrew Paverd:Eh, absolutely, yeah. And if you follow along the, the Twitter feeds of both Microsoft Research Cambridge, and the Microsoft Security Response Center, we'll, we'll make sure to tweet about any of the, the new work that's coming out.Nic Fillingham:That's great. Well, Andrew Paverd, thank you so much for joining us on the Security Unlocked Podcast. We'd love to have you come back and talk about some of the projects you're working on in a deep-dive section on a future episode.Andrew Paverd:Thanks very much for having me.Natalia Godyla:Well, we had a great time unlocking insights into security, from research to artificial intelligence. Keep an eye out for our next episode.Nic Fillingham:And don't forget to tweet @MSFTSecurity. Or email us at with topics you'd like to hear on a future episode. Until then, stay safe.Natalia Godyla:Stay secure. See for privacy and opt-out information.
Close your eyes, and imagine a world where booting up your computer wasn’t a susceptibility point for attacks. Imagine a Root of Trust that’s integrated into the CPU. Imagine all of your devices being protected against advanced attacks. Now, what if I told you there’s a cutting-edge processor that’s battle-tested for hardware penetrations, easy to update, and protects credentials, encryption keys, and personal data all at once? What if I told you it was already here, and your systems might already be using it?! Open your eyes, and get ready to be amazed! It’s Pluton, baby! Peter Waxman, Group Program Manager at Microsoft, joins hosts Nic Fillingham and Natalia Godyla in a tell-all about Pluton. Trust us, Pluton is sure to knock your SOCs off (that’s System on a Chip)!  Now that your eyes have been opened to a more secure system, we’d like to ask you to keep the volume down, because you’ve just entered the Library of Threats. While it may sound like inspiration for the next installment of National Treasure, you won’t find Nicolas Cage in this library (at least you shouldn’t). However, you will find Madeline Carmichael, MSTIC’s Threat Intel Librarian, whose movie-worthy title is just as impressive as it sounds. To be honest though, you might not find anyone in the library, as it bears more resemblance to Professor X’s Cerebro than it does your local hardcover sanctuary.   In This Episode, You Will Learn:     •What the Pluton Security Processor is and how it was created •The architecture of the Pluton Security Processor •What challenges were faced while bringing the Pluton Security Processor to life •The Root of Trust today vs. The Future with Pluton •The naming systems for threat actors, from periodic elements to volcanoes  Some Questions We Ask: •What differentiates the Pluton Security Processor from previous methodologies? •Why is the Pluton Processor better than what we have used in the past?    •What challenges lie ahead with the next steps around Pluton? •What has changed since Pluton was in Xbox to where it is now? •What tools and platforms does a Threat Intel Librarian utilize?  Resources: Microsoft Pluton Announcement:  Peter’s LinkedIn     Madeline’s LinkedIn  Nic’s LinkedIn      Natalia’s LinkedIn      Microsoft Security Blog:  Transcript(Full transcript can be found at Fillingham:Hello, and welcome to Security Unlocked, a new podcast from Microsoft where we unlock insights from the latest in news and research from across Microsoft's Security Engineering and Operations teams. I'm Nic Fillingham.Natalia Godyla:And I'm Natalia Godyla. In each episode, we'll discuss the latest stories from Microsoft Security, deep dive into the newest threat intel, research, and data science.Nic Fillingham:And profile some of the fascinating people working on artificial intelligence in Microsoft Security. Natalia Godyla:And now, let's unlock the pod. Hey, Nic, how's it going?Nic Fillingham:Hey, Natalia. I am good, I am excited. I've been excited for every episode, but I think this is the episode where we may be able to spin off into a major, major motion picture. I'm quite convinced that one of our guests, their story is compelling enough that a Nicolas Cage-style act, maybe even Nicolas Cage would be willing to turn this into a film.Natalia Godyla:Let's line up the two guests, and l- let our audience figure out which one is the next National Treasure.Nic Fillingham:First up, we have Peter Waxman, who's gonna talk to us about the Microsoft Pluton announcement from back in November of last year. This is a continuation from a conversation we had with Nazmus Sakib a few episodes ago where we talked about ensuring integrity at the firmware layer up and secured-core PCs, and now we're sorta continuing that conversation, deep-diving into what is the Pluton. Our Microsoft Pluton technology was announced in November. Fascinating conversation. And then we speak with?Natalia Godyla:Madeline Carmichael, who has a background in library science and worked in physical libraries, and now she is a threat intel librarian. So her title is MSTIC Librarian, she helps to catalog the different threat actor groups that we monitor. So it's a callback to a conversation that we had with Jeremy Dallman about tacking nation-state actors. Nic Fillingham:Yeah. So Madeline's job, apart from, uh, you know, one of the things that she does is she helps name these nation-state actors. And so we, Jeremy walked us through the, uh, periodic table of elements that is used to actually name who these nation-state groups are. So I just think that's fa- that's fascinating to go from a physical library and sort of library sciences into the deepest, darkest recesses of nation-state threats and nation-state actors. I- I think that is a Nicolas Cage vehicle waiting to happen, and I can't wait to go back into the cinema and we can sit down with our popcorn and we can watch National Treasure 7: MSTIC Librarian. This time, it's elementary? (laughs)Natalia Godyla:(laughs).Nic Fillingham:National Treasure 7: Threat Catalog- Catalog. Don't judge a threat actor by its name. No. Natalia Godyla:I see it. I know why you picked Madeline's. I feel like we probably need a little bit more help on that tag line, so if anyone wants to give us some feedback,, let us know. We are actively working on this script. Nic Fillingham:On with the pod?Natalia Godyla:On with the pod.Nic Fillingham:Welcome to Security Unlocked. Peter Waxman, thanks for joining us.Peter Waxman:Thank you, great to be here.Nic Fillingham:So this is gonna be the second of three deep dives we do on the sort of very broad topic of ensuring the integrity and the security of physical devices through things like protecting firmware, and obviously we'll expand upon that in this conversation here. Peter, you're joining us today to talk about the recently-announced Microsoft Pluton processor, so that, this is gonna be great. We're excited to chat with you. Nic Fillingham:Um, before we get into that, we'd love to ask. Tell us a little bit ab- about yourself. What's your job? What team are you in? What's the mission of the team? What's your day-to-day look like?Peter Waxman:Awesome, awesome. At Microsoft, I work in, uh, the Enterprise Security team, part of the so-called Azure application platform. Basically what we do broadly is build all the operating system platform and everything underneath. You can think about it as Windows, the operating system, you know, Windows that powers Azure. Even what powers Xbox and some of our other devices. Peter Waxman:And in particular, what I do is I focus on the operating system security and the low-level platform security that that operating system depends upon. Think about the hardware and firmware that our partners produce, to go make sure that that experience is completely secure. It protects our customers' data, it protects their identities, it makes sure that their application run with integrity and that they don't get hacked. And if they do get hacked, that we have an easy way to update and renew the system to get them in a good state again.Natalia Godyla:And so, we recently announced on November 17th the Pluton processor. Can you tell us about that? What- what is Pluton?Peter Waxman:Yes. Yeah. This is a big, exciting thing. It's something that we've been working on for quite some time. What Pluton essentially is is it's basically a security chip that lives inside of a larger chip. We call it basically the Pluton security processor, and this is like the heart of the security system in a PC or in a device. Peter Waxman:If you think about the security of a device, when you push power on that, when you push power on your laptop or computer, the, and the CPU comes up, one of the most important things is that the way that that system boots up and starts happens in a secure fashion. Because if it doesn't happen in a secure fashion, then it's very easy for bad actors to basically get in underneath and to root the system and cause all sorts of problems. Peter Waxman:So what Pluton is is basically this root of trust, the security processor that we, Microsoft, are integrating, and which is what we announced along with our major silicon partners in AMD, Intel, and Qualcomm, into the fabric of their products, in to the fabric of their chips. And so, by having that tight integration, it ensures that basically those chips and those products come up and boot in a secure fashion, and that we can then run Windows on this trusted foundation where we know the system is secure and basically we have, uh, much stronger footing with Pluton in the system going forward.Natalia Godyla:So what differentiates the Pluton security processor from previous methodologies? What were you using in the past? Why is this better?Peter Waxman:So traditionally in, uh, most PCs, the root of trust today is actually a separate chip. You know, very typically a discrete TPM. And that is something that lives on the motherboard as a separate unit, but it basically communicates over an insecure bus to the CPU. And the problem with that is that it just, it lends itself to all sorts of attacks. There's been a variety of ones that have been published. One of the common things that it's been known and in a published attack, basically there's one called TPM Genie. That bus, because it's insecure, even though the TPM chip itself may be highly secure, the system overall is not. Peter Waxman:And so, attackers can go in with very inexpensive hardware, a logic analyzer, $50 worth of equipment, and go and basically intercept and alter the communications between the CPU and the TPM. And end up basically, you end up with an insecure system as a result. You could actually be booting malware in the firmware. You could basically be booting with exploits all through the boot chain, and Windows wouldn't know about it. The customer's data and experience would be compromised as a result. And so, by moving the root of trust into the CPU die, we're basically taking a whole class of attacks out of the scope, resulting in a system that is more secure overall in terms of how it comes up and the foundation. Peter Waxman:It's also something, though, that one of the challenges that exists with the existing roots of trust is that they're very hard to update. Like other components in the system, right? They have their own firmware, the firmware can have vulnerabilities, and in fact, there have been notable vulnerabilities that have existed in TPM firmware. And when we look and see across the inventory of Windows 10 systems out there, there's actually a very large number of TPMs that are running out-of-date, unpatched firmware.Peter Waxman:Uh, as a result of having Pluton integrated into the CPU and having tighter control of it from Windows, we can leverage the decades of experience and billion-plus endpoint reliability that we have in Windows Update to offer customers the ability to much more easily and automatically update firmware on the root of trust of the system. If there's ever any security issue that we find, we can very quickly get an update out. We can also, importantly, update with new capability, so as new scenarios come online, where customers want to take advantage or applications want to take advantage of this root of trust, we have the ability to add that capability to Pluton in a easy, quick ability through Windows Update. Natalia Godyla:So what challenges did you have with bringing this security processor to life, with bringing it to PCs, in particular with the partners and OEMs that we were bringing it into the market with? And- and what challenges still lay ahead with the next steps that you have around Pluton?Peter Waxman:Yeah, so there's plenty. I mean, there's a- there's a tremendous, uh, satisfaction that we have and, you know, came to the point where we have been able to announce with our major silicon partners that we're bringing this to market. But I'm humbled by it, but at the same point we still have a ways to go before this comes to market. And to continue really in seeing to the vision, which is really to enable Pluton everywhere and to be ubiquitous even beyond PCs and- and gaming consoles and- and IoT devices.Peter Waxman:So- so a lot more work to do. Working with the ecosystem is something that takes a lot of time. It's been a tremendous effort, it's been several years in the making just to get to this point where, you know, we're far enough along with our partners that we can announce it, that we feel confident around landing these products. Both with the silicon partners that we announced, as well as with a range of PC OEMs that have been with us on this journey over the last year.Peter Waxman:We're at a point, though, because, you know, we're basically taking Microsoft technology and integrating it with our- our silicon partners, it's our silicon partners' products that are the ones that will bring this to market on OEM devices. They are not yet ready to announce sort of their particular timeframe intercepts, so unfortunately I won't speak to exactly when products land. But, you know, they are coming, folks should stay tuned. Peter Waxman:And when you think about Intel or AMD or Qualcomm chip, kind of the rule of thumb is it takes three years to go from the time that you start the design to the time that you have the chip in hand. So that's a long process. We're well away, well along that path in terms of where we're at, but it's lot of, obviously, detailed architectural work. Peter Waxman:We're excited about, uh, the product finalization and also thinking about sort of the next set of steps and next silicon products for integration. But it's- it's a huge effort across a range of companies to- to land something like this.Nic Fillingham:Is the goal to be integrated across the entire silicon spectrum in terms of consumer, low-end, affordable consumer devices, all the way through to secure e-work stations, uh, and sort of everything in between? Or it specifically a solution for more security-conscious, sort of enterprise customers?Peter Waxman:Great question. Yeah. No, so this is important. We see this capability as something that just is a fundamental security property that needs to be there on a modern device. We have seen, we've all seen how over the last 10, 15 years there's just been an increasing amount of sophistication, not just in software attacks but in attacks that basically deal with low-level aspects of vulnerabilities in firmware, hardware attacks that exist. You can get up to nation-state stuff, and we see things, whether it's in the Snowden leaks or particular instances of nation-state attacks, that are taking advantage of, say, firmware vulnerabilities.Peter Waxman:But it's more common that than. I mean, there are criminal networks that have exploited UEFI components in PCs to basically connect PCs to botnet networks to cause a variety of- of issues there. There continue to be, on a week-in, week-out basis, month-in, month-out basis, vulnerabilities that are reported that exist in a variety of firmware components or new hardware disclosures that exist. Peter Waxman:So it is something that is cross-cutting, it's something that is not just an enterprise issue. It's something where, you know, this raises the security of all devices, and is basically something that the average consumer has a right to expect of their device. That expectation Peter Waxman:Absolutely needs to be there from the lowest end consumer device to the highest end enterprise device. We... And, and Microsoft just committed to that. Natalia Godyla:So with Pluton becoming a new industry gold standard, I'm sure that also means that it'll become a target or a goal for hackers to try to break into. So, what are the challenges for hackers? What would they need to overcome in order to actually hack to Pluton processor in a, in a hypothetical situation? Peter Waxman:Yeah, it's a good question. I mean, there's certainly, especially in the research community, there's a lot of established, uh, research and techniques that folks do to, uh, break into hardware products. I mean, we've seen that certainly, like, going back to the Xbox days, right? There's, uh... One of the things that's interesting about sorta the consumer gaming security space is that in order for the adversaries to thrive, they're not necessarily a criminal network, they're not a nation-state, and they need to share information so you can kind of observe them more easily. But there are techniques and capabilities that folks have addressed and, obviously, with Pluton we're trying to ensure that we are targeting a bar that makes it very challenging for them to attack the system. Peter Waxman:It is one, though, we're never gonna say that there's any perfect security system, and so you have to design your system to be renewable. You have to allow for the fact that they're going to be, gonna be issues that are gonna be found and make sure that you can update, you can patch, and also that you have defense in depth. So, if a hardware measure is defeated, you have something backing that up. We feel confident about, uh, Pluton just in terms of its, it, it is battle-tested. Peter Waxman:This is something that we started on this journey 10 years ago. We've continued to invest in the capability and we're not done investing in the capability. We will continue to harden and strengthen it over time. But it's, you know, we're, we're talking about super cool equipment that a variety of folks'll go over to try to glitch and figure out what timing abilities does an attacker have to figure out if they issue a, a 20 nanosecond pulse on exactly this pin and exactly ti- this time at boot can they glitch the system to cause a, a, or, say, a crypto operation or what have you to basically fail. Peter Waxman:These are the rates of attacks that come into a scope when you get into hardware security and, so, we've got a bunch of super bright folks that are experienced in this space, but, uh, we'll be interested to see how the threat actors respond and... It's also important to note that Pluton, we don't trust in the system, there's a critical security component, but it's not the only security component, right? The whole stack of, uh, security that, you know, st- stands on top whether it's an OEM device and their firmware or in Windows itself or in applications. These all matter, too. Peter Waxman:An application can still have a vulnerability in it that is remotely exploited regardless of Pluton being in the system. And, so, you've got to look at the whole system from a security perspective to make sure that, uh, we're continuing to drive security across, up, and down the stack. Nic Fillingham:And, Peter, I assume, uh, Microsoft, as well as the actual silicon manufacturers, you know, they're actively gonna be pen testing, uh, the Pluton processor over time, right? So, as Pluton is defined and as it goes into production and as it actually gets into the hands of, of customers, there'll be a continual effort on behalf of Microsoft and, I assume, also the silicon partners, too. Keep it secure and, and see if we can hack it ourselves to, to deter and find any potential vulnerabilities and address them. Is that part of the process?Peter Waxman:Absolutely. Absolutely. Nic, so, Microsoft, the history that we've got with Pluton, we have both ourselves and involved third parties in doing hardware penetration tests, hard- hardware hacking on it to assess its strength. We have a, a long history of working with our hardware partners on hardware security and working with them on basically issues in firmware and hardware in their silicon. And, obviously, for the particular partnerships, both parties, you know, in this case Intel, AMD, and Qualcomm, are fully aligned with us in ensuring that their security teams, our security teams, red team and pen test teams, and external evaluation that, basically, we get as much eyes on this to find any issues before anyone else does and, hopefully, to not find anything, which has been the case to date. When we do, to basically respond and, and react to, uh, accordingly with our partners. Natalia Godyla:And, what learnings did you have so far from the days in which you put Pluton into an Xbox and now? Like, what have you changed in the processor for the PCs for this new announcement? If, if anything?Peter Waxman:We've evolved in a number of areas. I think that one is that just the application of it is different somewhat in the PC than it is in an Xbox than it is in an IoT device. So, for example, TPM functionality, which we talked about earlier is something that we don't need a standardized TPM in the Xbox. It's all sort of vertically integrated. Stack, we do things that are similar to a TPM, but we don't need that capability. But in a PC, that's a standardized functionality that exists in pretty much every PC today. And, so, there are capabilities that we've added to be able to, say, support that from a firmware perspective and where needed to add additional hardware blocks.Peter Waxman:We have advanced. There's places where it's just a matter of hardening the design that we have in Pluton. So, some amount of resistance to physical attacks that we've increased over time. And, it's also, you know, supporting newer capabilities that may exist in, in the industry. If I think back to Xbox days, the expectations around crypto key lengths, for example, right? We didn't have as many crypto algorithms or quite as long key lengths. We supported, say, in the, you know, early implementations of HSP versus today. Now that we have quantum crypto creeping up on us over the next 10 to 15 years, right? There's a much higher focus, for example, on longer crypto key lengths to make sure that we can maintain resistance until we get to sorta implementation, more common implementations of post-quantum crypto algorithms. Peter Waxman:So, some examples of places where we have just evolved and, um, you know the way Microsoft views it the Pluton-based, the, the architecture and design is something that we evolved for all end points and, so, you'll see, for example, that the Pluton is in the latest Xbox series X and S that we announced, came to market with, and launched in November is a more advanced version, right, based upon that newer capability set then what was there in the Xbox One. So, as I mentioned, continue to sort of update this technology and continue to make it available through these range of markets.Nic Fillingham:I want to ask about the architecture of the Pluton security processor. When it goes onto the actual CPU die, is it going to be a tax on the CPU? Is it, or is it sort of occupying such a trivial amount of sort of transistors and, you know, storage elements that you're not gonna know that your computer is Pluton powered? It's just gonna be happening silently and completely invisibly in the background.Peter Waxman:Yeah. That's r-, that's right. It is, from a power perspective or sort of any other aspect from an end-user, you're... Basically it's a small component when you think about it in relation to a modern SOC or modern CPU. It's not taking any relevant amount of power that's at all gonna be noticeable from the device perspective. It's basically this hidden component inside the SOC, system on a chip, complex that, uh, is basically working on your behalf ensuring you have a much higher security experience as a result, but you will not notice it being there. That's right. It's basically invisible. Nic Fillingham:And, and just circling back to that Xbox comment, so, so I've got an Xbox One, uh, here at home. It's the Xbox One S.Peter Waxman:Yep.Nic Fillingham:So, there is a version or a precursor to the Pluton on my Xbox. Is it Pluton v. 1 or is it pre-Pluton? How should I sort of think about that? Peter Waxman:You've got Pluton. You've got Pluton.Nic Fillingham:I've got Pluton? Peter Waxman:You got Pluton.Nic Fillingham:Yeah. Peter Waxman:Yes.Natalia Godyla:(laughs)Peter Waxman:(laughs)Nic Fillingham:Can I get a sticker? Can I get a sticker to put on my Xbox that says you got Pluton, baby?Peter Waxman:I will get to work on that, Nic. I love the idea. I love the idea. I think... I... Your t-shirts and stickers. I think that's, you know, that may be the, uh, the holiday project coming up. Nic Fillingham:And, then, so, moving forward, at some point, when I'm buying a new piece of computing, whether it's a laptop, whether it's an IoT device, or I get something else with a CPU inside it, I'm gonna want to look for probably a Pluton sticker or a Pluton badge or something that lets me know that the CPU or the SOC contains the Pluton architecture. Is that, again, part of the vision for Pluton?Peter Waxman:It's a great question. I don't think we've come to a conclusion on it. I'm not sure that we're gonna get to the dancing Intel guys in their, uh, clean suits, uh, commercials on T.V.Nic Fillingham:That's a, that's a callback to, like, is it the 90s? When they do that? That was a long time ago.Peter Waxman:(laughs) Yeah. That's, that's showing my age there, perhaps. Nic Fillingham:Natalia wasn't born then. She doesn't know what that is. Peter Waxman:(laughs). Natalia Godyla:Right over my head. Peter Waxman:(laughs)Nic Fillingham:(laughs) But, I mean, in terms of as a consumer, or a potential consumer, or even just a, you know, an employee at a company, do you envisage that it'll get to a point where I'll have, you know, an opportunity to buy a Pluton secured device and a non-Pluton secured device and so, therefore, I'm gonna wanna think about my needs, my security needs, and make sure I'm getting that Pluton secured device or, again, maybe to what you said earlier, it's just gonna be completely invisible, completely integrated into the silicon? You're not gonna worry about it, but you're just gonna know that there's, there's a higher grade of sort of fidelity and security on that device because of the architecture in the CPU. Peter Waxman:Yeah, I mean, our goal is really to get to that point where it's ubiquitous and it's just there. I mean, it's, again, if we're gonna provide, uh, customers with the level of security that is required in today's day and age, we've got to get to a point where this is like oxygen. It's everywhere. It's just a common ingredient that exists. We have to work with our ecosystem. We have to basically work to a path where, you know, we get there. It's not on the market yet. It's gonna take some time. There will be points in time where it's a journey to get there and not every system is, is certainly gonna have it, but our vision is this just needs to be everywhere. Peter Waxman:It's something where, you know, we're doing this not to make money off of this thing. Not to basically drive specific scenarios. Not to charge and up-prem as we talked about earlier for enterprises. This is about how do we make sure that everyone from consumers to enterprises to you name it has something where we're taking the last 15 years of hardware and systems security, hard learnings, and bringing it and modernizing the PC space based upon those learnings. Nic Fillingham:How did you come up with Pluton? I had not heard Pluton before I plugged it into Wikipedia, which is the font of all knowledge and it tells me that it is an igneous intrusion... No. No. It is a body of intrusive-Peter Waxman:(laughs).Nic Fillingham:... igneous rock. So, how'd you get Pluton, but, maybe more importantly, tell me some of the names that you considered, but didn't go with? Can you-Peter Waxman:(laughs)Peter Waxman:... can you let a few cats out of the bag? Proverbial cats out of the proverbial bags? Natalia Godyla:Most important question. (laughs)Peter Waxman:So, this one, Nic, I think we're gonna have to put the pause button on the recording-Nic Fillingham:Ahhh.Peter Waxman:... and I actually have no good answer nor do I have a great joke to go, uh-Natalia Godyla:(laughs)Peter Waxman:... to go, to go make fun. You know, so, it's, like, code name/buzzword that we use publicly. It's one word. It sounds cool. Nic Fillingham:It does. Sounds very cool.Peter Waxman:It's not named by anything else. And, uh, it's... If you think about hey, this thing is going to set the direction and do something leading, it's, like, a north star that's out there. Sounds cool. I don't know what it means. Nic Fillingham:(laughs)Natalia Godyla:(laughs)Peter Waxman:I didn't even know it was an igneous rock until you mentioned it, honestly. But, uh, yeah. Exactly. I...Nic Fillingham:It is an igneous intrusion.Peter Waxman:Igneous intrusion. I stand corrected. Natalia Godyla:(laughs) Peter Waxman:God. I'm gonna have to go look up that 'cause that, that's kind of freaky and scary. Natalia Godyla:I feel like that's the best answer.Peter Waxman:(laughs).Natalia Godyla:It sounds cool. Nic Fillingham:It sounds cool. That's totally-Peter Waxman:It's authentic. Natalia Godyla:(laughs)Nic Fillingham:Yeah. That's totally fine for it to sound cool. I did wonder if there might have been something a little bit more sort of esoteric and, and deep to it, but I'm totally happy with it sounding cool. We'll have to, we'll have to go and talk to some of your colleagues to see if, uh, maybe Dave Weston can let us in on a few, uh, names that didn't make it that we could, we could make fun of on another podcast episode. Peter Waxman:Yeah. Microsoft Bob was one option, but it was taken. So, uh...Nic Fillingham:(laughs) Peter Waxman:Yeah. No. Dave will be good to, uh, get history there.Nic Fillingham:Peter Waxman, thank you so much for your time and for joining us. And, uh, I would love to have you back on the podcast on another episode to discuss the history of Xbox security and maybe mod chips and Xbox hacking and all that cool stuff that we all did in the early 90s. Oh, the early 2000s-Peter Waxman:(laughs)Nic Fillingham:... I should say. Peter Waxman:Awesome. Awesome. I really appreciate it, Nic. Natalia, it's been an awesome discussion so thank you very much.Natalia Godyla:Yeah, thanks for being on the show. Natalia Godyla:And, now, let's meet an expert from the Microsoft security team to learn more about the diverse backgrounds and experiences of the humans creating AI and tech at Microsoft. Natalia Godyla:Hello, Madeline Carmichael. Welcome to the show. Madeline:Hi, thanks for having me. Natalia Godyla:It's great to have you on the show. I have never talked to a threat intel librarian before so let's start with that. Can you, can you tell us about that role? What does your day-to-day look like? How did get into Natalia Godyla:... becoming a Threat Intel Librarian.Interviewee:Yeah. I mean, I can pretty safely say you're among good company in not having met someone with that job title (laughing). I get a lot of really interesting reactions to the title. And, to be honest, it's kind of self-styled (laughs), so it's not like an official Microsoft HR title. But that's the one I go with for my day to day function and what I actually do. So, basically, I work as part of the Threat Intel team in the Microsoft Threat Intelligence Center and as a Threat Intel Librarian for them. And that means I'm sort of responsible for organizing a nation-state threat actors that we track and supporting the end-to-end business process that enables the team to do that as efficiently as possible.Interviewee:So, recently, I've added being a MITRE attack evangelist to my description and my role. So I look at how we can integrate that framework into our workflows and how that can help us do more with our data to support internal workflows. But also how we can share better Intel with our partners. And the MSTIC team sort of tracks nation-state actors, primarily. There's a little bit of wiggle room around human-operated ransomware. It's becoming a, a more concerning threat and we're, we're onboarding some of that. We currently have more than 270 groups on our radar and that's between named groups that we, we name after the periodic table of elements.Interviewee:So, so when we speak publicly, you'll hear things, uh, named after that. And then we have what we call dev groups, which are sort of the pre-stage, it's for our internal tracking and to keep, keep things in order. But we don't tend to discuss those publicly. Yeah, we do like security detection, analytics, um, response capabilities for Microsoft end customers. And that kind of entails providing threat intel to Microsoft and defender teams across the company, and then out to customers through security products. So I originally started as a, well, thought I was going to be a librarian and probably a public librarian at that. I was doing that degree and there was an option to do, uh, an internship or a co-op for credit, not a requirement, but I found an interesting job posting.Interviewee:So took a chance and applied for it and got it. And that was with a research library for the government of Canada. And that was great. I really, really enjoyed working there, and actually, ended up finishing my last (laughs) two degree credits distance while I was still working. That kind of led to moving on to a team that my role was doing aggregate reporting and sort of trend analysis a little bit for the executive leadership at the org. And from there, just got interested in the actual cybersecurity analyst part of the team, and eventually, moved over to that, which was where I got the skills that kind of transitioned into my role at Microsoft.Natalia Godyla:I'm just going to un- unpack some of the roles there and some of the skills that you're, you're bringing to role as a Threat Intel Librarian. So in the research library, when you're saying that you got into data reporting, what, what were you reporting on?Interviewee:So that was mostly incidents that have been tracked by that team during the month or the quarter. And so it was just kind of aggregating that data in sort of human-readable format that could be sent up to executive leadership. So they were aware of kind of the high level trends that were happening.Nic Fillingham:But, you, so when you were studying, you said you, you found a job posting, you said it was an internship, is that correct?Interviewee:Yeah, co-op internship. However you want to call.Nic Fillingham:Got it, a co-, a co-op, and that was with the government of Canada/Interviewee:Yep.Nic Fillingham:And is it accurate to say that was sort of more of a traditional librarian style role? You, you are physically in a building that had a lot of sort of printed stuff or am I like way too old school and antiquated in my thinking (laughing)?Interviewee:No, it was kind of in the middle of that. There was a physical library, and yeah, definitely more towards the traditional end. Slightly untraditional, I guess, in the sense that it was like a focused collection. So it was specific to the type of research that, that group was doing. But, otherwise, yeah, books and cataloging and, uh, organizing that.Natalia Godyla:Why cybersecurity or how were you exposed to cybersecurity? Was it part of the research that the library had or was it just that subsequent roles brought you closer and closer to the field?Interviewee:Mostly the sort of subsequent role is getting closer and closer. It feels pretty serendipitous when I look back at it now. Like I didn't intentionally set out for a career in cybersecurity or Microsoft or where, where I am. I, uh, did a presentation a couple of years ago for a conference, uh, in the UK that's run by a woman at Microsoft and it's called TechHer, more, more like TechHer. So I did this presentation at TechHer, which is a, a conference run by Microsoft UK. And it aims to kind of give women more networking opportunities and sort of more visibility into technical roles. And during that presentation, I, I called myself an Accidental Threat Intel Analyst.Interviewee:At the time I was still in that analyst role, more the, the Threat Intel Librarian role. And it's kind of true, like I never intended for that. Accidental is maybe giving myself too little credit for taking some, some opportunities that presented themselves (laughs). But, yeah, it was just kind of each pivot kind of brought me one, one step closer and I thought it was really interesting. And I've been lucky to work with people who are really engaging and their passion for it is contagious. So, yeah, I guess that's why I stuck around.Nic Fillingham:So what do you do as the Threat Intel Librarian to expand the collection of knowledge and data and, and papers and content in a particular direction? Who, who are your customers and, and how do you go about expanding that collection?Interviewee:My customers, I guess, or my, my user base would be the threat analysts on the team. And the collection of data is their analytic output, essentially. So it's less curating new collection and less providing resources as it is organizing the output that they're producing. So we have a, a knowledge base that holds all of the threat intelligence that the team produces. And the aim there is to organize that in a way that makes it more friendly for capturing data, but also, um, produces more usable output for downstream users, whether they be in Microsoft as other security teams or Microsoft customers through security products.Nic Fillingham:And what tools or sort of platforms do you use, you know, this knowledge base? Are you, is it SharePoint or is it some other sort of more secure encrypted storage system? I mean, uh, maybe you can't talk about it, but, but what sort of in, in a general sense do you, are your tools that you're using day in, day out?Interviewee:So that's changed over the years since I've been here. I've had a number of iterations where we store things, we, we're using, uh, DevOps at one point and kind of mashing that into our scenarios. But we're now using a proprietary knowledge base that's being developed by a dev team out of ILDC.Natalia Godyla:So what big goals do you have around the library that you are maintaining, building? What's, what's next for you to optimize? What are some challenges that you're trying to tackle?Interviewee:Well, yeah, so the, the nature of tracking nation state threats and like threat actors is that capturing the relevant threat intel means you often end up with a lot of data that's constantly evolving based on what the actors are doing. It's hard to keep tidy. So the ultimate goal, I guess, is to make our knowledge base as organized as possible to enable as much automation as possible. The threat analysts do a lot of repeatable pivots or queries. And those are really important for, for maintaining an ongoing awareness of what the, the threat actors are doing. But a lot of that can be codified and then made into a repeatable process where they just have to like check in and make sure it's functioning accurately.Interviewee:And then that allows time for them to do the really clever stuff that takes nuance and a human sort of intuition and experience with tracking for actors to do well. Not all of it can be reproduced by a computer. So as much of the sort of day-to-day stuff that we can automate as possible, that's, that's great. And we do that by having well-labeled classified data that's organized, and yeah, we can feed it to an automation pipeline and then let the analysts do the fun stuff.Natalia Godyla:So speaking of classification, we, we chatted with Jeremy about how we came to the names of some of the threat actors. I know you mentioned we use the periodic table. What was the impetus for that? Why are we using the periodic table and wha- what's going to happen after the periodic tables run-up?Interviewee:(laughs) Uh, well, that was in place before I started. So I, unfortunately, can't take credit for (laughing) why it was chosen. I think it was probably chosen because it's a, a, a ready set of names that are easily identifiable to the general public. You can kind of say we named things after periodic elements and most people will know or have some familiarity with that. So there's some, not really branding, but that kind of familiarization so that if you hear a name like that, you think MSTIC and Microsoft. It's also not rooted in a specific culture, really, so there's not any cultural connections or connotations that you need to worry about for applying a name. It's going to be used publicly and associated with Microsoft (laughs), so.Nic Fillingham:One of the questions we asked Jeremy was, is there a logic behind why one particular group would be given a particular element? Like, you know, are all the inert gases, are they, are they a particular continent or something? Or were they all discovered in the 2000s? Is, is there, is there any logic or is it, is it... because I think the, the joke we made with, with Jeremy was whether or not there was a, a big periodic table of elements against a wall? And then there was a bucket of dots (laughing). And as a new group comes out, you grab a, you grab a dart and you throw it at the wall. Uh, where are you in that continuum?Natalia Godyla:It's funny the second time around too.Interviewee:Yeah, I mean, honestly, I wish that was the case. It would be pretty cathartic, I think. But, no, there- there's no logic to the, the name choices we decided or my predecessors decided not to add that layer to the naming. So they're meant to just be just a name. We're, I think, careful as Microsoft about what kind of associations or what we mean when we say, like, we, we choose what we say carefully. And I think it was intentional not to associate that sort of, um, this type of name means this origin for an actor. We, we wanted to have that level of abstraction still.Natalia Godyla:There are more groups, though, don't you track more groups than there are elements in the table? Is that, am I right there?Interviewee:Yeah, so we have two types of groups. The ones that have element names are what we would call sort of permanent groups, or it's a permanent name. And that kind of is just the level of, uh, awareness we have for the group. So it's a more mature understanding of the threat actor that has that name. Um, we have a second type of name and we, we call them dev groups, um, dev for development. And it just means they're, they're in development and they're not as fully f- fleshed out as the element names. So it gives us a little more flexibility to kind of label clusters of activity without having to do as much rigor b- behind that sort of is that cluster and what its scope and breadth is.Interviewee:So there's definitely cases where multiple dev numbers or dev groups will merge into one named element group as we develop more of an understanding about who the threat actor is. Um, yeah, so I think we have over 185 dev groups on the go at the moment, and then 89 element groups. And that will probably change very quickly. So the numbers are not actually that useful (laughs), uh, uh, long-term, but yeah. It, we, we have more dev groups because they're easier to spin up and faster and they're, they're meant to be precursors for the named groups. But as, as you say, there are not that many elements. So we, uh, we'll be running out rather soon (laughs). I'm not sure what's going to come out.Nic Fillingham:You'll be into the theoretical element-Interviewee:Yes.Nic Fillingham:... category, genre. What's the one from, uh, Avatar? Unobtainium or something?Interviewee:Yeah, yeah, I think that might be it (laughing).Nic Fillingham:Was that right? And then there's, what's the one that's bonded to Wolverine skeleton? That's, that's a made-up one too, isn't it?Natalia Godyla:Oh, you have an, uh-Nic Fillingham:Adamantium, Adamantium (laughing).Natalia Godyla:... wealth of knowledge about this (laughing).Nic Fillingham:Yeah.Interviewee:We recently actually added another name schema and they're named after volcanoes. I don't know if that came up in your conversation with Jeremy, but as we put more focus on tracking human-operated ransomware groups, we thought they're distinct enough from the nation-state groups that we would have a separate schema for those. So there's some, some volcano names that are out Interviewee:... they're now, and it's the same kind of idea where dev numbers still support both names. And as we develop maturity, it, of awareness on a group, if it's a nation-state, it'll get an element and if it's human-operated ransomware, it gets a volcano.Nic Fillingham:You know what? I probably should've asked this at the tippy-top of the conversation, but why do we name these groups? What is the value in assigning a name and then actually sort of publicizing the name of that group? Where, where is the value to threat hunters to analysts to customers? What- what's the rationale behind this?Interviewee:Yeah. So, I guess it's mostly for consistency. It's, it's kind of a language of its own. And you use language to communicate, so having a name and being able to explain what that name means is important. So, one of the other things that our team does is write activity group profiles. They go along with alerts in security products. Interviewee:So, a customer might get an alert and they'll get this, this document that contains context of what that means for them, and that will include things like the TTPs that that group uses, some of their infrastructure, or like malware that goes along with it, and context that kind of explains their typical motivations or their typical targeting. Interviewee:So if you're in an industry that is a, a usual target for that group, it might make sense for you to say, "Oh, yeah. Like, it makes sense that we were targeted, it makes sense that this alert is hitting our network, or our endpoints." Interviewee:But it is also useful to know if you're an outlier in that circumstance. That might mean you pay more attention to it because you're not a typical target for that group. But yeah, so having a name is just a, a way to kind of say, "We mean this group," and here is the context that goes with it, and it's a consistent message.Natalia Godyla:What other ways are customers benefiting from this library? So, you noted that the alerts will have some of this context that you've been gathering. What other features or capabilities are based on the library?Interviewee:So, yeah, it's our awareness of the group long term. So, it allows us to kind of see what we would expect of them. We, because we have this body of knowledge built up, we can then see quickly if a tactic or a technique that they're now undertaking is brand now. That's kind of a departure from their normal M.O., that's more interesting. It's useful context. Interviewee:Yeah, for Microsoft as well as customers, we use our own TI to help defend ourselves. And, yeah, I guess it's just a, a way to kind of contextualize what is happening with IOCs or indicators of attack. They're kind of distinct bits of information that help you detect or protect or respond to a threat.Interviewee:They contextualize indicators of attack or IOCs, and those, those can be really s- like, small bits of information that help you detect a threat actor. And just having an IP address doesn't really tell you a lot, so that's useful to kind of have that explanation that goes with it that says, "This IP address is used by this group in this way," and that informs how you respond to it as well, depending on the, the attack slide, is useful for how you mitigate that. Interviewee:And that's a, a big part of why we're starting to add the, the MITRE ATT&CK classification to our data as well. It's a clearer language or repeatable way of describing something to your customers. And the customers as well have started to use attack labeling in their own data sets, so it's a good way to kind of match things up.Interviewee:And you can layer customer protections that have been mapped to the attack framework with detections on our side that have those attack techniques labeled. And when you layer those on top of each other, you can find gaps really easily and find how they might need to improve their security posture in a certain area.Interviewee:If, say, its reactor uses a certain technique and that, that customer has a, a gap in detections in that area, they can go, "Oh, well, we are a typical target for this group. We're not super well secured in that area. Maybe we should focus our investment there."Nic Fillingham:So, is it accurate to say that naming these groups and sort of building and maintaining a profile on them allows both hunters and analysts and then customers to better understand where they may or not be a target, and then therefore, how their security strategy should evolve?Interviewee:Yeah, definitely. Yeah. Natalia Godyla:(laughs) Nic Fillingham:Cool. I got my head around it. I must admit, the very first time I read a, a blog post from Mystic and I, I saw, you know, the name, like, "Here's the name of the threat actor and here's what other industry groups sort of name them," I was like, "I don't get it. Why, why are we naming them?"Interviewee:(laughs) Nic Fillingham:But, I, I got it now. So, thank you so much.Interviewee:(laughs) Cool, glad that came through. (laughs) Nic Fillingham:I'm glad that this podcast exists, exclusively for me to, to get my, (laughs) get my questions answered. Natalia Godyla:(laughs) Interviewee:(laughs)Nic Fillingham:Hopefully someone had a similar question and we, we helped answered them. Thank you.Natalia Godyla:So now that you've been in the cybersecurity space for several years now, come to a role that feels like it marries a lot of what you've studied and done throughout your career, the cybersecurity and library are coming together in the name. What comes next that is... Does this feel like it's a merging of the worlds or is there something you want to do after this, either in the cybersecurity space or not?Interviewee:That's a great question. Yeah, I wish five-year planning came easier to me. (laughs) Natalia Godyla:(laughs) Interviewee:Although in, in the world of COVID, I don't know that anyone can plan that far ahead. But yeah, I, I don't know. And I think because I got sort of sidetracked from my original public library path, I haven't really thought about how I would go back to that. Interviewee:I mean, libraries are becoming much more digital now anyways. It's a great way to serve more content to your patrons and your, your, your users in the world of e-readers and eBooks and podcasts and things like that.Interviewee:Libraries procure that kind of content for their users all the time, but yeah, I don't know. I don't, I don't know what's next. I mean, I'm happy where I am. So, yeah, stick here for a little while. Nic Fillingham:Madeline, one of the questions we'd like to ask in, in this part of the podcast is what from your personal life, your hobbies, your interests outside of work, so first of all, what are they? And second of all, do any of them, do you bring any of them into your day job?Interviewee:Yeah. I mean, I feel like this is where your assertion earlier that I broke all of the librarian stereotypes will fall down, because I do love to read and I have two cats. Um... (laughs) Natalia Godyla:(laughs) Nic Fillingham:(laughs) And you just travel round to libraries with your-Natalia Godyla:(laughs) Nic Fillingham:... with your cats and your, and your book bag? That's all you do? Interviewee:Uh, yeah, yeah. I mean, if the cats were allowed in the library, that would definitely be something.Natalia Godyla:(laughs)Interviewee:But I think library tourism is a very underrated area. Expedia should look into that. Nic Fillingham:And apart from reading, cats, and visiting other libraries, is there anything else you're willing to divulge?Interviewee:(laughs) I don't know that a lot of it actually makes its way into my day job. Baking is another hobby, but we're not in the office, (laughs) so I can't really share that with anybody. Nic Fillingham:What's your favorite baking show to binge? Are you a Great British Bake Off fan?Interviewee:I am. Since moving here, I've definitely started watching that.Natalia Godyla:(laughs) Nic Fillingham:Have you thought about entering? Do you wanna be a contestant?Interviewee:I did actually consider it at the end of this year's series, but I haven't got up the nerve to actually apply yet, and I don't know that I could take the pressure of having to figure out all of those (laughs) different baking techniques without a recipe. (laughs) Natalia Godyla:What is one of your favorite books of all time? I was gonna say, what's your favorite booK? But I feel like that's just an impossible question to answer, unless you have one.Interviewee:I, so I generally read fiction. That's my primary genre, but that kind of covers a lot of different (laughs) sub- sub-genres of fiction.Natalia Godyla:(laughs) Interviewee:I think my go-to answer for my favorite book is usually Anna Karenina by Tolstoy. (laughs)Nic Fillingham:In the original Russian? Interviewee:Of course, yeah. No. (laughs) Nic Fillingham:(laughs) Natalia Godyla:(laughs) Interviewee:No. Yet, I should say. Um-Nic Fillingham:There, there's different translations, right? Is-Interviewee:There are, yeah.Nic Fillingham:Which one do you like? Interviewee:It's by Richard Pevear and Larissa Vol- Volokhonsky, I think. I'm probably not pronouncing her last name very well. But yeah, it's, it's a great book. And it's long and you have to flip back to the, the list of character names every five pages or so and every character seems to have five names.Nic Fillingham:(laughs)Natalia Godyla:All the diminutives. Yep. (laughs) Interviewee:Yes, yeah, (laughs) precisely. Nic Fillingham:(laughs) Interviewee:Uh, but it's good. I, I just, it has always stuck with me as a book I really enjoyed. Natalia Godyla:Well, thank you, Madeline, for being on the show. Interviewee:Yeah, it was great to speak with you guys. Thanks for having me. Natalia Godyla:(singing) Well, we had a great time unlocking insights into security. From research to artificial intelligence, keep an eye out for our next episode.Nic Fillingham:And don't forget to tweet us @msftsecurity or email us at with topics you'd like to hear on a future episode. Until then, stay safe.Natalia Godyla:Stay secure. See for privacy and opt-out information.
CCI: Cyber Crime Investigation. Another day, another email attack - something smells “phishy” in the network. *Slowly puts on sunglasses and flips up trench coat collar* Time to go to work. Just how easy is it for someone to steal your credentials? Because once they’re stolen, and sold for pocket change, it’s open season. Homoglyphs, drop accounts, email forwarding… is it any wonder billions of dollars have been lost to BEC (business email compromise)?Join hosts Nic Fillingham and Natalia Godyla for a fascinating conversation with Peter Anaman, Director and Principal Investigator of the CELA Digital Crimes Unit, as they unpack the cybercrime section of the Microsoft Digital Defense Report to see what these phishers are up to. Scott Christiansen joins us later in the show to recount his journey to security and his role as an Adjunct Professor for Bellevue University's Master of Science in Cybersecurity, along with some great advice for choosing security as a profession.       In This Episode, You Will Learn:    •The difference between consumer and enterprise phishing •The types of people and professions that are usually targeted in cyber attacks  •How putting policies on backups and policies to protect the organization in place will help prevent digital crimes •The four categories of the internet: the dark web, the surface web, the deep web, and the vetted web   Some Questions We Ask:   •What would an example of credential phishing look like? •What is the end goal for phishers? •How are phishing and business email compromise techniques leveraged during the pandemic? •What patterns are being seen when it comes to credential phishing? •How do you use ML to classify whether a bug is security-related or not? Resources:   Microsoft Digital Defense Report:    Peter’s LinkedIn   Scott’s LinkedIn   Nic’s LinkedIn    Natalia’s LinkedIn    Microsoft Security Blog: Transcript(Full transcript can be found at Fillingham:Hello and welcome to Security Unlocked, a new podcast from Microsoft, where we unlock insights from the latest in news and research, from across Microsoft Security, Engineering, and Operations teams. I'm Nic Fillingham-Natalia Godyla:And I'm Natalia Godyla. In each episode, we'll discuss the latest stories from Microsoft Security, deep dive into the newest thread intel, research, and data science.Nic Fillingham:And profile some of the fascinating people working on artificial intelligence in Microsoft Security. If you enjoy the podcast, have a request for a topic you'd like covered, or have some feedback on how we can make the podcast better-Natalia Godyla:Please contact us at or via Microsoft Security on Twitter. We'd love to hear from you.Natalia Godyla:Hi Nic. Welcome to Episode 13.Nic Fillingham:Thank you, Natalia. Uh, welcome to you as well. I'd just like to say, for the record, I like the number 13. I'm embracing 13. Do we know why 13 is unlucky number? Is there ... Is it just superstition?Natalia Godyla:There are a lot of theories. 13 people at the Last Supper, that's part of the reason. 13-Nic Fillingham:At, really?Natalia Godyla:... steps to the gallows.Nic Fillingham:I'd, I think this is baloney. I don't think-Natalia Godyla:(laughs)Nic Fillingham:... this is real. I think-Natalia Godyla:I think-Nic Fillingham:... 13's a great number. I think we should celebrate it-Natalia Godyla:You know what? That's a, that's a good approach. Let's do it.Nic Fillingham:And we should celebrate it-Natalia Godyla:With jokes-Nic Fillingham:With a joke (laughs). So, before we started rolling, we were lamenting the fact that there are very few, if any, like, true, sort of security, cybersecurity-flavored jokes. So, we sort of created some, or we, we've evolved some. Do you wanna go first, Natalia? 'Cause you've got a joke that I've not heard. So this would be, in theory, a genuine reaction. Do you wanna give me your joke?Natalia Godyla:Yeah. Ready?Nic Fillingham:Yep.Natalia Godyla:What's a secret agent's go-to fashion?Nic Fillingham:I don't know. What's a secret agent's go-to fashion?Natalia Godyla:Spyware.Audience:(laughs)Nic Fillingham:Spyware. Yes. That's all right.Natalia Godyla:Wow. Didn't-Nic Fillingham:It's okay.Natalia Godyla:... even try for a chuckle.Nic Fillingham:I did. No, I genuinely did. I was like-Natalia Godyla:I barely got a smile, guys.Nic Fillingham:Aw, I was hoping to like that one. It just-Natalia Godyla:(laughs)Nic Fillingham:... spyware, yeah. No, it's okay. So, you've heard this already, but the audience haven't, and I know that they're all gonna be absolutely cracking up when they hear this. So, what do you do when your pyramid gets infected with Ransomware? You encrypt it. That's pretty good, right? That's pretty good.Natalia Godyla:I've got a new one. We're gonna try-Nic Fillingham:Okay.Natalia Godyla:... a new one.Nic Fillingham:I'm gonna try and laugh. Like, I'm gonna be in the right frame of mind for, if it is funny, I'm gonna try and laugh. You ready? (laughs)Natalia Godyla:I like that little "If it is funny." All right-Nic Fillingham:Well.Natalia Godyla:Why doesn't Superman fight cyber crime?Nic Fillingham:Why?Natalia Godyla:Because he's scared of cryptocurrency.Nic Fillingham:Oh, no, no, no, no, no, no, no, no. Okay, so it's a joke about. It's a jo, no, no we're gonna pull this one apart and we're gonna fix it.Natalia Godyla:Right. Right.Nic Fillingham:So it's a word play on cryptocurrency. So, it's gotta be something like, Superman's laptop, no that's not it. But we're gonna work on this.Natalia Godyla:Strong start.Nic Fillingham:If you're a, a dear listener of the podcast, if you think you can make this Superman joke work for us, let us not. or hit up on the Twitter's MSFD Security.Natalia Godyla:So do we wanna tell everyone about this week's episode?Nic Fillingham:(laughs) I, I guess we probably should. On today's episode, we speak to Peter Anaman who is gonna talk to us about business email compromise. This is the fourth of five conversations we're having on the podcast to cover content from the MDDR. Peter explains to us the difference between sort of general phishing in the consumer email space, and phishing and email compromise in sort of sort of business corporate world, and also what the attackers are doing once they do compromise a business email account. Make sure to follow along at home by downloading the Digital Defense Report And then after that, we speak with-Natalia Godyla:Scott Christiansen a senior program editor at Microsoft who as he says it "is the security conscience for our company". So, he does a lot of work on the software development lifecycle and ensuring that we are delivering secure code, that we're adhering to our policies and standards around what it means to have secure code. And, in addition to all of that, he's a professor so he talks to us about the cybersecurity program that he's part of and it's a great conversation.Nic Fillingham:It is. On with the pod.Natalia Godyla:On with the pod. Nic Fillingham:Peter Anaman welcome to the security unlock podcast. Thanks for joining us.Peter Anaman:Thank you for inviting me.Nic Fillingham:Well, we'd like to start the podcast off with getting our interviewees to give us a quick introduction to who they are. Obviously we'd love to know your title but more uh, interestingly is tell us about what you do uh, day to day. What's your, what's your job look like?Peter Anaman:So my name is Pierre or Peter Anaman and I work in the digital crimes unit in the Microsoft [inaudible 00:05:08] Organization, which is the legal group. And within this group I'm part of the Global Strategic Enforcement Team, and we currently are focusing on BEC or Business Email Compromise. As regard to my title, Cyber crime Investigator, so I focus on developing cases that we then either pursue with a civil lawsuit or, you know, or to identify the thread actors, or we develop cases that are then subject to a criminal refer to law enforcement where we believe the thread actors are located. So, that's what I do on my day to day basis. As far as looking at prints, looking at intelligence, dark web data to try and see how the criminal, online criminals are using different tools in order for us to try and be ready and up to date. Nic Fillingham:That's an amazing title. I'd love to have that on a business card.Peter Anaman:(laughs)Nic Fillingham:So is your background law enforcement? Are you a lawyer? This might be a very uh, broad question but how did you get to where you are? Peter Anaman:So I started off pursuing um, once I finished my high school I always wanted to be a lawyer, and so I pursued legal studies and went to law school in the UK. And when I finished law school I, I had a, uh, a passion for pursuing like legal, um, law enforcement related activities, and the law and police was one but I heard the army had a very stringent course in France, and so I pursued a full month uh, accelerated course to become an officer in the French Army. And uh, so, and thereafter I was a Lieutenant. I had to leave but always had a purs, um, a passion for enforcement and from there I ended up working in a law firm trying to combat online piracy as well as different types of cyber crimes. Peter Anaman:So, it, it included piracy but it was also, child sexual abuse material where you know, we uh, support the law enforcement where we can. And that just developed. And I developed skills. I did amass this in information security to learn some of the tools, how the internet works, and just learned what I needed to and was curious. I spoke with a lot of experts that they taught me so many things on the way. And now I ended up working in this amazing organization.Nic Fillingham:On today's episode in this discussion, we're talking once again about the, the Microsoft Digital Defense Report, the MDDR which came out uh, in September of, of this year of 2020. And Peter, you're here to talk to us about a section or, or part of the state of cyber crime which is called phishing and business email compromise. You, you contributed heavily to this report. Could you just sort of tee us up, if, if, if you've not heard about the MDDR, the Microsoft Digital Defense Report and you're sort of you know, interested in downloading it and learning more, tell is about this section of phishing and business email compromise. What, what's the scope of this section and what, what are you gonna learn in it?Peter Anaman:Phishing has been um, you know with a Ph for those who don't know, involves where, typically involves where people [inaudible 00:07:57] are sent emails to people, and once in the inbox entice you to click a link, you know to upgrade, update your password or something of that nature, increasingly is being related to themes like news, like Covid-19, or election related. And when you click the link you go to a site where they ask you for your credentials, and once they have your credentials then they in most cases, may have access to your account. Unless you've got two factor authentication or some other security measures. Peter Anaman:And so, this section what we try to deep dive, is try to explain the different types of cases that may fall in that, in that category of online crime. And what I mean by that is you see from the sections there's one on credential phishing, there's a second which is more based on BEC Business Email Compromise, sometimes called CEO Fraud and we can speak about it a bit later. And then there's a third category which is really a combination of first two where the thread actors use credential phishing and then lead to some kind of fraud, financial fraud.Natalia Godyla:So wha, what patterns are you seeing when it comes to credential phishing? How does this manifest in an attack? What would an example of credential phishing look like? Peter Anaman:So when you look at each of these sections, the three of them, I can provide a little bit more depth. And so, in the first instance, credential phishing, as I mentioned earlier, it would be when a person would receive and email claiming to be you know, security department or a, you know, some h, highly important thing that they have to do, and when the person clicks the link, they are then sent to a webpage which looks like the, the legitimate office 365 login page as an example. And when they enter their credentials, the source code of that webpage has a form and the form has instructions. And those instructions are, when someone clicks submit, collect information in the username and password, and send it to what we call a drop account. Right? It's like an email address that collects the information submitted on that page.Peter Anaman:Now, we know this because through our investigations, we analyze you know, a p, I think we're on about ten [inaudible 00:10:06], hundreds of thousands of URL's every day to determine if they are phishing or not. And so we have seen how the in, information submitted from the email and from that email, what they do in some instances, in credential phishing is that they know that some people, like researchers will submit dummy information. So what whey do is they do a, a check. Right? They take the credentials and try to impersonate someone sent connected to the account, using some con, uh, they call it an SMTP checker, it's a, as in to keep the protocol for sending email. And so they check the credential and it works, they know it's valid. If it's not valid, they get rid of it.Peter Anaman:And then, once it's valid, we have seen like literally in minutes, it can lead to what we call BEC and our [inaudible 00:10:51]. So that's credential phishing essentially. But boldly the three differently areas we're seeing these credentials being used, we see them being sold on the dark web for very little. Because then other people can use it to send spam for example, or unsolicited commercial emails. They could use it to look at the person's account and steal confidential information, or business email compromise. So, that's how credentials are used typically.Peter Anaman:We then move to BEC and CEO fraud. There it's uh, I think most of the time, some people like to use BEC to include phishing but it's really a different type of activity. And the reason they use business email and compromise, is that this activity is targ;eting companies. And the reason is, it's another way of stealing money from the bank, right so to speak. And what I mean by that is that they've realized, the criminals have realized that companies have processes in place. Right? So for example I wanna b, I wanna pay for a service. Well it goes to procurement, and it goes to accounts payable, and they make a, a payment. Peter Anaman:Well, understanding this kind of almost a supply chain, right? The criminals have realized that, s, Peter Anaman:If they can monitor for wire transfers or transactions, they can like take over that conversation and redirect the payment to a different account. And this is how it could work based on what we've seen. So, as I mentioned, you have credential, they then have access to your account. When they have access to your account, in most cases we see two things happen. One, they add a forwarding rule. So they add an inbox forward- forwarding rule which says if you receive an email and in the subject or the body, you see accounts payable, invoice, USD, EUR, so different keywords that are related to a transaction, forward it to this email account. In other cases, what they do is they say forward it to an RSS folder. So a folder in your account and so then they will access your account and that specific folder to get the email messages which makes it harder to identify who they are, right? Because if they have an email or someone accesses that email. Peter Anaman:So once they add the forwarding rule and messages are sent and they find an email about the payment due, what they do is they look at who are the parties and depending on who, who is the person receiving the money, they'll get rid of them on the chain and create a homoglyph domain name. A homoglyph, it's like the Egyptian times, right? Something that is made to look like. It impersonates another domain name. For example, an I becomes a one. Right, or O for Oscar becomes a zero. So it's a slight change. And what they do then is that they have to use the same name as the person who they've removed and they continue the conversation. And at some point they say, hey, my account has changed. Updated PDF, this is our new bank account.Peter Anaman:Well because the people on the chain have been part of the chain, they think is legitimate. And so they make changes to the payee, to the instructions. And then the money is moved to a different account. It's just terrible when you see how much money has been lost. And if you read all the reports, you know, it's in the billions of dollars that have been lost this way. And that's why BEC has become very, very important to tackle as a type of crime. Peter Anaman:Now the third category, we said was a combination. And the reason is that in BEC, the second category, there are cases where it's almost like a stakeout, right? They see a company because they go to a website like, uh, the city has to make public, all the RFPs, you know, orders that they have to do 'cause they have to be public. So they see who may be bidding for a contract. And then they'll impersonate that person and try and get access to the payments for that government contract as an example. So that doesn't use credential phishing, right? It's, they're just looking for public information in order to understand what relationships are and to take over a transaction. Fascinating stuff, you know. Someone could make a movie out of how these people operate.Nic Fillingham:And is BEC the sort of end goal for the phishes? So for example, is phishing in the consumer space, the harvesting of, of credentials then being used to launch and mount, uh, BEC attacks in order to actually make some money?Peter Anaman:So I think there is a way we can distinguish between consumer and enterprise phishing. So the difference between sort of a, a spray concept, which is for consumers, just try and get as many accounts compared to the enterprise, the business email compromise, where it's more targeted. And the difference is that when you create a new Hotmail or Outlook or Gmail account, the systems know it's new, right? When I say it's new, is that if you were to send me an email from, right, I would know it was created yesterday. But if it started to send emails to like a lot of, 200 people is highly suspect. But if you were able to get a person who's had the account, like let's say for 10 years, right? Well maybe that's not a anomaly because the person has lots of friends. They have lots of contacts, right. The, it looks like a real person. And so it's more likely to go under the radar when it comes to detection. And those could be some of the benefits of using compromised consumer email accounts. Just one example, there are many others.Peter Anaman:On the enterprise side, what we've seen for example in some of the attacks, is that the people who are being targeted typically within the category, right? We see a lot of executives, for example, in the C-suite that'd be being targeted. We see a lot of people in the accounts department, which have been targeted. We see directors being targeted because these are people who can authorize payments. They're not looking to send an email to a person who cannot help them, unless maybe it's an executive assistant who then can give them access to the inbox of the C-suite. Peter Anaman:Now in my presentation, I've spoken at times of dark web and I think I'll just put a sentence behind that. You know, dark web is a word that is used often, but in this context, I'm just speaking about places where people sell, conduct activities associated with criminal activity. The web is divided into four categories from my lens. One is the surface web, which is indexed like through search engines. The second is called the deep web. Those are websites that are either password protected like an online forum, where you have to register an account before you get in or a dynamically created website. So for example, a new site where the content changes, changes on a regular basis. So that's a deep web, it's not index. One of the biggest parts. Peter Anaman:Then the dark web is really tall, right? That's where you need a specialized search engine, you have to use, go to dot onion websites and that's a different category, dark web. Then you have the vetted web. The vetted web are websites where in for you to get access you need to be vouched. Which means that another criminal has to say you're a bad guy, and or girl. And so then you will be able to access it. And it's a way for them to try and trust each other. But in my context-Nic Fillingham:It's the, it's the Twitter blue tick of, of the bad guys.Peter Anaman:Yes, they're trying, they're trying, they're trying. Uh, but [inaudible 00:18:17] all of them. So, you know, for, for what that matters.Natalia Godyla:One other section of the Microsoft Digital Defense Report that you had covered was the section on COVID-19 themed phishing learners. So can you talk a little bit about how these techniques for phishing and Business Email Compromise were leveraged during the time of the pandemic and are continuing to be levered?Peter Anaman:So one of the, one of the patterns or trends we've noticed is that often the criminals change their attack mechanisms or the way they send messages based on lures which are relevant to a group of people in a specific time. As an example, we saw the same with you see it with, uh, elections or sport games or something to do with a celebrity. In this case with COVID-19 at the beginning of the year, we started to see a change and he came from a specific and came in different people were doing it, but we saw it more naturally with one group. Where we were tracking them for mid-December on the activities they were conducting, phishing activities they were conducting. They were using for example, financial statements, or they were using bonuses or different lures about finance and then all of a sudden they changed and they started to use COVID-19 bonus as a lure where they would say, "Hey, click this link to find out about your club COVID-19 bonus."Peter Anaman:And so when people click the link, it was sent to an Office 365 login page, and they submitted their credentials. A lot of people submitted their credentials from the logs we've analyzed because they believe that it was something that was relevant for them at that time. And that was part of the lure. And after a few months they changed, we were able to technically counter what they were doing and they moved to a different method of attack. It's just using, using the time.Peter Anaman:We just recently saw it with elections, for example, the same thing, the US elections. And we saw there were, there were some groups who had modified how they presented the email to people in order to encourage them to click the link and lead them to a phishing page. So the COVID-19 lures are something that we've noticed. It's part of a broader theme related to, uh, societal events, which are criminal's trying to take advantage of to increase the possibility of people clicking a link, right? It has to be believable. And it has to be a sense of urgency.Natalia Godyla:Do you ever think we'll preempt the societal moments? So if there's some big moment happening, we can assume that a cyber crime would leverage that societal moment as a lure and so we could plan ahead?Peter Anaman:One thing which would be difficult is as a company, we have a wide array of customers and we want all our customers to show up the way they want to show up, you know, without having to try and be someone else and not authentic. And with that in mind, it really, and even a step further, these people, right? They work for different organizations and in different organizations, they have different cultures that they have different ways of working. If you look at, for example, a manufacturing company where maybe IT may not be at the forefront, what the way they interact with IT will be very different to if you went to a startup, a tech startup, where that's what they do most of the time, not manufacturing, right? And so when we have such a wide array of customers and we've got governments, right, we got governments from different countries, some like each other, some don't. We have banks, we've got, we have different types of customers and Microsoft, all of a sudden becomes the protector, right? Because criminals are targeting banks, but they're our customer. So they rely on our security as well.Peter Anaman:So when we go back and speak about lures and things, these are things that we have to as cyber-crime enforces, we have to understand it happens. And so as we build technical measures, we have to implement technical measures that are adjustable and can, can change based on patterns it's observing. So I think the way to attack it is always to have this kind of different measures that are working together and leverage artificial intelligence and machine learning models in order to help us distinguish between different types of criminal activity and protect our customers. If that makes sense.Natalia Godyla:And what is our guidance to customers on what they can be doing to help prevent against these attacks?Peter Anaman:One is always to have good policies in place within the company, right? So that all employees are aware about how to make sure the devices are up to date. Don't pick up a USB on the street and put it in, you know, uh, make sure internally there are policies on backups, make sure you've got an online and offline backup, right? So you have to have policies in place that help protect the organization. The second part is to work hand in hand with their technology providers, right? So for example, if you work with Office 365, make sure that we have something called a Secure Store, a Secure Score. that's Secure Score is based on experience. We can say, hey, maybe if you have, to have a better score put MFA, Multi-factor authentication. Some of your users allow forwarding, block it. [inaudible 00:23:40] make sure it's admin can only authorize forwarding, right? Or off. 2.0, make sure that, uh, consent has to be from the admin. So there's a secure store that it helped them really implement in a much more secure environment, which will be frictionless. Number three is to have regular tests Peter Anaman:... with any organization. So that, I mean, that could be part of the policy, but typically is not always. Where you have fishing simulations, which are taking place, right? So that you can start to e-, keep the education at the forefront because we're all very busy and sometimes we forget. And I think four is that we have to work, we have to look always to use technology to advance the way you work forward. And what I mean by that is that companies need to think about the digitalization of their work processes. And what I mean is, uh, I mean, this may be a little bit off, but investigating some ransomware cases.Peter Anaman:For example, recently we saw that part of the problem is that some customers have old infrastructure on-prem, for example. And so that is what is being attacked. And once they get into that, then they can pivot and move laterally elsewhere into the organization. So I think digital transformation is by looking at your processes overall, by saying, "Are there ways we can modernize in a way that creates a better security landscape?"Nic Fillingham:Well, thank you for your time today. Again, we were, we were talking about the Microsoft Digital Defense Report, which is available to download for free. We'll put the link in the show notes. Peter Anaman or Pierre Anaman, thank you so much for your time.Peter Anaman:Okay, thank you very much. Be safe.Natalia Godyla:And now let's meet an expert from the Microsoft Security team, to learn more about the diverse backgrounds and experiences of the humans creating AI and tech at Microsoft. Hello, everyone, and welcome back to another episode of Security Unlocked. Today, we are joined by Scott Christiansen, who is a Senior Security Program Manager at Microsoft, as well as a Professor at Bellevue University. Thank you for joining us, Scott.Scott Christiansen:Well, thanks for having me. I appreciate it.Natalia Godyla:I'm really looking forward to this conversation. So, so let's kick it off by just giving a little bit more context behind those two roles. Can you tell us what your day and, and night look like as a program manager and professor? What do you do? What does your team look like? What do you teach?Scott Christiansen:Yeah, absolutely. So let's start with Microsoft, that's the thing that takes the majority of my time. So (laughs) I work in our customer security and trust group. And, specifically within that, our security engineering group within customer security trust. And then, more specifically, I work in our data analytics and insights team. And our group, as a whole, our security engineering team, is responsible for ensuring the company meets the software development life cycle, operational security assurance, policies and requirements that we have. As for any shipping software that we have to ensure that what we're shipping out meets our own internal, um, security standards and our internal security rigor.Scott Christiansen:Which then is tied to plenty of different external security compliance objectives and things like that. So that's kind of a mouthful, but we help ensure that the company's delivering secure code is kind of the nutshell. Or as we like to say, we're kind of the security conscious for the company. We have security teams throughout the products and then throughout the organization. And we're the conscience that comes through and says, "Is everybody doing everything they can be doing? And are there areas where we could be doing better and, you know, how can we help in that space?"Scott Christiansen:And so what we started doing is we started pulling in all the bugs across the company. So we've got like 700 different Azure DevOps repositories where engineers are storing work items and working with. And they generate roughly about probably 50 to 60,000, uh, new work items every single month. And so we suck in all that data to one gigantic data warehouse and we perform kind of analytics on that. That's really branched out to kind of work streams that I very specifically work on. One, I've spoken a little bit externally about this, where there's a blog up on the Microsoft blog site. I've spoken at RSA this past year and it's kind of their machine learning work that we've done with security bug classification.Scott Christiansen:So we pulled in all of the security bugs to this one spot. We said... and some of them are labeled as security, some of them aren't. And we took a look at that and we said, "Well, are there any that aren't labeled as security that should be labeled as security?" So about four years ago, probably, we started a little hackathon project trying to answer that question. And, uh, it's been a small project kind of throughout time with that. But, ultimately, it turned into a product that we've put together where we built a machine learning system, uh, that accurately classifies, uh, these bugs and says, "Hey, this pool of bugs is security and this pool of bugs is non-security."Scott Christiansen:And then for the, the pool of bugs that it says it is security, it will, um, say, "Hey, yeah, these particular subset of those bugs are critical security bugs. These are important security bugs, or these are some other particular severity with that." And we've had just unbelievable accuracy with that. So that's one of the things that I work on. Yeah, so we've got that model built and we're in the process of really, uh, we've got it built. We've classified all this data that we have within the company, and now we're in the process of making that more operational, so the engineering teams can take advantage of it. And then, in turn, finding a way to take that and spend it externally, probably through GitHub.Scott Christiansen:Uh, that's kind of the target that we're looking at, but so external customers and just the security industry as a whole can kind of take advantage of this auto classification piece. I spend a portion of my day doing that. The other portion of my day is kind of around this, this compliance report and GitHub bot. A really incredible code analysis tool. Used to be called [inaudible 00:29:11]. And it does just a phenomenal job at finding software vulnerabilities. And it's our team's job to kind of get that deployed within the company. And right now with getting static analysis stuff rolled out i- is the biggest priority. So that's pretty much what I spend my day on.Scott Christiansen:And the evenings, like you had mentioned, I'm a master's level cybersecurity professor at Bellevue University, uh, specifically, in their online cybersecurity program. And there I teach a few different classes, but most specifically I teach their masters in, um, architecture and design.Nic Fillingham:Thanks for that intro, Scott, uh, oh gosh, I've, I've written down like four questions coming back to, I think, one of the first things you just talked about in your day job, if we can call it that, your Microsoft role, how do you use machine learning to classify whether a bug is security related or not?Scott Christiansen:It started as this, as this summer hackathon project, and it was just a few of us, myself, uh, one of my colleagues, Alok Kumar and one of our other colleagues, Naveen [Nurenja 00:30:09] sat down and said, "Hey, are we missing anything in this space?" And none of the three of us were, were data scientists by any means. Alok had a little bit more an understanding experience with some of the machine learning work. And so we sat down and we go, "Who are the big hot tents in July?" And I started chewing through this problem and I was an expert in the security space. And so I said, "Well, well, those guys were going through and they were looking to see if they could find a machine learning model that might kind of work to help us solve this problem."Scott Christiansen:I went through and I did manual sampling of the bugs to determine if there was actually an issue there or not. So we went through and took a couple thousand bucks that were taken as security and looked to see if we had any misclassified or misidentified bugs there. And then we took a bucket of the bugs that were not classified as security, like another 2000, 3000 random sampling of bugs. And said, "Are there any security bugs in that space that we're missing?" And so we found discrepancies in, in both spaces. And so clearly the things that aren't showing up on the security radar are potentially a problem. The, the good thing is there's a good side to this whole story is that engineers fix bugs regardless if they're security bugs or not.Scott Christiansen:So the stuff that we found that didn't necessarily show up as a security bug was still getting fixed and it was getting fixed within a, a good SLA. So that was good, the right thing was happening, but it wasn't necessarily maybe showing up on everybody's radar. And, more importantly, it wasn't necessarily showing up on a radar where a security assurance person can come say, "Hey, I see you doing some security work over here. Maybe I can give you a hand and I can help you out with that.2 And the, the same was true for the space where we saw all of these security bugs or things that were tagged as security bugs, but they weren't necessarily security related.Scott Christiansen:You know, engineers are wasting kind of these trimmed down SLA fixed times for these, you know, supposed security bugs that aren't there. And so we're spinning up all this excitement around, "Hey, oh, here's the, the security bugs that come in and you have to fix these things." But they're not actual security bugs, and so you're just kind of spinning your wheels on that and, and wasting available engineering effort. So we started building our own machine learning algorithm kind of around this. And we started kind of doing this manual assessment and said, "Okay, out of these bugs that are security, can we find clusters of bugs that are misclassified?"Scott Christiansen:And so, eventually, we did that and it took us a while, it took us a good probably year and a half to come up with, what we would say, was a really kind of gold standard training dataset. We had this big block of bugs, uh, roughly about 300,000 bugs that were classified as security and ahead with the right security severity. And we were confident in those classification numbers. And so that's what we used to then train the model. So as we're going through this, and we got about to that point, we said, "We really need data science expertise." We hired, uh, Mayana Pereira and she's our data scientist for the project. And she's absolutely fantastic.Scott Christiansen:She found error rates associated with the data and how flexible we could be as error information potentially got introduced to our training dataset. She's shifted the algorithms that we've used a couple of different times, and we are light years beyond where we were thanks to kind of her joining the team, uh, and joining the project. And so, yeah, it's been about a four year journey, probably.Nic Fillingham:So just to clarify this, so the machine learning model is simply looking at the title of the bug. It's not looking at like Reaper steps or any other data. It's just, what is the title of the bug?Scott Christiansen:Yup, yup, that's correct.Natalia Godyla:So the courses that you're teaching are around infrastructure and the work that you do and Microsoft is around software development. So how did you get into security? What have you done within the security space? What brought you to these particular domains within security?Scott Christiansen:So I used to actually live in Omaha. I'm not from there, originally from North Dakota, part of the small cluster of people that, that, in this world, that are from North Dakota. But I met my wife up there and we moved down to Omaha. I restarted kind of, kind of my education once I went to Omaha into computer science. I went to school there, I got a job, and eventually, I started working at an architecture engineering company. I say it's a small company, it was a 1200 person company, but it was, at the time, it was the fourth largest architecture engineering company in the, in the US. So it was decent sized.Scott Christiansen:Being a small company, you get hands-on with a lot of different things. And so I'm going to school, I'm working, I'm starting to run all the infrastructure components that, that we have within the company. And we've got like 13 different offices in the US. We started to expand internationally, so I got a lot of exposure in that space. As I'm going to school, I'm trying to figure out exactly what kind of discipline of IT I want to do. At that time, it wasn't necessarily development. I like the Microsoft products, I like server products, I like Linux products. It was really the, the infrastructure stuff. And so I started getting into networking, and then I kinda got bored with that.Scott Christiansen:And so then I kind of went to systems administration of Windows stuff. You know, that one was where I was thinking my focus was going to go. And then I kind of got bored of that. One of the unique things about Omaha is it has a really large, uh, department of defense presence down in Bellevue, Nebraska. They've got an air force space and they have strategic command that's down there too. And one of my professors happened to be a security person that worked at StratCom down at the base.Scott Christiansen:And he was really into security and he kind of taught us some security stuff. And I was like, "Whoa, this is kind of like the Jedi, Sith type of cool, you know, dark hacking. This was before like hacking was like super cool like it, like it is now. It was just kind of this thing, but it's was like, "Hey, you can get software to do things that the software developer didn't expect to do." I'm like, "This is kind of interesting. It's got like the prankster type of thing, right?" And you get this creative mind going and you start going, "I want to do security." So I'm working at the architecture business and I said, "Hey, I'd really like to shift my role into security."Scott Christiansen:So I started doing some security stuff for them, but it's not really necessarily a high target type of business when they said, "Hey, you know, if you're ever looking for something, we're looking for a lead in our incident response group." And, and so shortly thereafter, I moved over and I was the lead for the incident response team for, uh, TD Ameritrade for a number of years. And TD Ameritrade absolutely has targets, they have, not, uh, not only normal criminal targets, they've got nation Scott Christiansen:... state attackers and anybody that's looking to try and steal money an- and hack into large financial enterprises, so that was a really exciting job and we did a lot of really exciting, cool things there, and some neat stuff happened. And then one day, I, I got a call from our, uh, sort of VP of security engineering at the time and he said, "Hey, we really need some help over in the software assurance space." And so I moved over onto that team and wrapped up my dev and my code view chops, and started doing kind of code review and code analysis.Scott Christiansen:And, specifically around that time, we were getting into the mobile app space, and so that's where I really focused my effort, was the kind of mobile applications and ensuring we had security coding practices with that. And then, and then, eventually expanded to kind of, to, to the rest of the enterprise. So, I was working at TD Ameritrade during the day, and I was teaching the one location at night, and then teaching online in between that. Scott Christiansen:And then, I was writing some, uh, the local, um, security groups, too, like the OWASP Omaha, I was president of that for a little while. I was the president of Nebraska InfraGard for a little bit. So pretty active in there, and, uh, Microsoft reached out to, out to me, and said, "Hey, look. We've got this opportunity, and we'd like to talk to you about it." And it's Microsoft, right? So I'm not gonna say no. It's like, you know, some of the smartest people in the world working on these kind of world-changing problems. Scott Christiansen:And I came out, and I will say it took the third different position at Microsoft before I finally actually moved out to Redmond and started working for Microsoft full time. I had two different opportunities tha- that didn't work out. So anybody who's ever interested in working for Microsoft, don't give up. There's enough people here and enough opportunities, I'm sure the right opportunity exists out here for you. And, and clearly it was, because this was ... Eventually when I came out here to do this work, this was absolutely the right fit for my skillset, for the company, and it was this kind of perfect blend, and I, I wouldn't think of anything different beyond that. Scott Christiansen:I absolutely love what I do, and I'm now in a role where I have an opportunity to ... You know, I'm not just securing an enterprise or securing a company. I'm part of, uh, really changing a- around the world as a whole. So it's this really, kind of wonderful opportunity and wonderful role that, that I get to do and these kind of global changing types of things that we ... problem solving, I guess, that we get to work on within the company.Natalia Godyla:I love the context and I can absolutely vouch for your statement about Microsoft. I came to Microsoft after the second roll, um, so going inside Microsoft or having the inside out perspective, I now understand the sheer size of Microsoft and the fact that you just keep trying. If the right fit is there, it'll happen. But your story seems to really have started with a professor who highlighted security as an opportunity. So is there any connection between that professor and your desire to go into teaching? How did the professorship start?Scott Christiansen:Very good question. I was pretty active in the local Omaha security community with the different groups, and there was a guy named Ron Warner, and Ron's a good friend of mine, still is a good friend of mine, and he was very active in the community as a whole. And, around the time that Bellevue University was standing up their cybersecurity program, Ron was there, and he called me up, uh, he was standing up the program. He was the director of the program at the time. Scott Christiansen:And he said, "Hey, look. We're standing this thing up, and I know you've had some experience teaching at ITT Tech." And I started teaching at ITT Tech, 'cause I graduated with my master's degree. I was still, um, friends with some of the professors there, and they said, "Hey, you should come teach for us." And, interestingly enough, I decided to teach for one very specific reason. I wasn't a very cohesive public speaker, and it was a skillset that I really wanted to grow and develop, and I thought. "Wow. A, there's no way for me to be a better public speaker than to go up day and day in front of a group of people and try to deliver a message, and I'm not just talking about something at that point in time. I'm teaching them something, so they have to come away with knowledge after that."Scott Christiansen:So it was really like a self-growth thing in a space that I felt like I had some level of expertise. Over the course of time, I really started to, to, to develop kind of a rapport, and almost a character, like y- y- you'd put a hat on say, "Okay, this is, this is my teaching hat. This is what I'm gonna go do," and you deliver something that's interesting and engaging. And there was a personal growth component with that, because I'm this old guy by this time. I'm married and I've got kids. I don't have a lot of extracurricular time on my hands, but I have all of these students. Scott Christiansen:It was, uh, it was a scattering of, of male and female students. So I could start to take new ideas and present them as seeds to the students. So like, "Hey, I wonder if you did this," or, "There is an interesting security tool. Do you think you could do this with it?" And I could pique their interest and they go out, and the next week they came back and they're like, "Hey, look at this thing that I did." And so then we all got to learn together with them. That was really, really personally rewarding to be able to do that, to help people learn, but also to see the feedback and me, individually, grow from the knowledge that they were presenting back to myself and back to the class, too. So it was really incredible.Scott Christiansen:And security is hard. It's not an easy discipline. It's not an easy space. It covers the gamut of everything. If you think about security kinda holistically that, you have all these engineers building all of this technology to do thing, security is trying to understand what they did and figure out where they went wrong. So, I don't have to get a lot of people excited about security anymore. They're already excited, 'cause they've started the program. There's definitely some level setting that you have to do, and let them understand kind of what the space looks like, versus what they think it's gonna look like. Scott Christiansen:Everybody think they're gonna come in and they're gonna be a pin tester and they're gonna make millions of dollars and find all these vulnerabilities, and that might be the case for some people. I mean, there's bug bounty programs out there, where people are making significant amounts of money. But there's a space than that, and that's a very specific subset of everything that you can do in security. There's a lotta opportunities for lots of other people to do lots of different things. So I'd like to help do that, too. Scott Christiansen:But more importantly, I'd like to help the students understand how to properly secure things. There's a lot of misinformation kind of in that space, or people have misguided expectations on how to secure specific things. There's a definitely a right way to r- to do things and a wrong way to do things, and so that's one of the things that I feel I probably contribute the most is saying, "Here's a right way to do this." But sometimes, if you have some knowledge or, or you have that background already, i- the online experience can be very successful for you, or if you're just really good at ... you don't mind asking questions.Nic Fillingham:I love that you said if you find yourself not succeeding in an in-person environment, go check out online and see if that's the right thing for you, and, and the inverse. That's fantastic advice. Well, Scott, is there anything you wanted to plug or, uh, point people to before we let you go? Any sort of resources, blogs, communities you like?Scott Christiansen:Besides assessing that the machine learning model is the right tool, or the machine learning that we built right now is the right tool for external customers, we're doing a lot of our own, individual assessment. You know, Microsoft has gone down this awesome path of responsible AI and ethical AI. So, wh- We're no different to that process. In addition to seeing how well the model does within this outside Microsoft, we're also running it through the gamut. Scott Christiansen:So we've taken it through, um, our legal resources to say, "Here's our model." You know, "If we were to release this thing tomorrow externally, would you be okay with it? Here's the data that we used. Here's the data owners that own the data that we're using. Do you think it's okay with them that we've built this model and it does these things?" We've got security teams now within the company that do, uh, this responsible AI and security AI work, and we've talked to them through the risks associated potentially with our model and, and what the model could do. Scott Christiansen:That whole security AI space is really new, so it's interesting for a security team to come out with this security classification model and then kind of go through all those reviews. We're in the process of starting to work with some security AI pen testers now within the company, so people that in their specific skillset is attacking these AI and, and ML models and finding vulnerabilities and flaws kind of associated with that. So we're engaging with them, uh, to do that.Scott Christiansen:So we're doing a lot of different work kind of with that. And, again, that's all because we've trained this model on a non-public data set. So, if we expose the model externally, we wanna make sure that it's not gonna expose any of this non-public information to the rest of the world. If all this turns out and it fails, so far, it looks like it's not, but if it does, then, you know, being a responsible engineer in this space, we have to go get public data to do this.Scott Christiansen:And if we trained it with public data, that would be fine, but it's taken us three years to kind of get to this particular point to build up this kind of reference data set. It's gonna take that long externally. And so what we wanna do is try and see if what we have is, is good enough to put out there, but, uh, do it in absolutely the most responsible way for Microsoft and our engineers and our customers that we possibly can. So if there's any plug, i- it is that plug and that responsible AI is super, super important, and we're doing our best to kind of adhere to those goals.Nic Fillingham:Well, Scott Christiansen, thank you so much for being on Security Unlocked.Scott Christiansen:Yeah, absolutely. Thank you so much for having me. I ... Uh, it was really rewarding. I really appreciate it.Natalia Godyla:Well, we had a great time unlocking insights into security from research to artificial intelligence. Keep an eye out for our next episode.Nic Fillingham:And don't forget to tweet us at MSFTsecurity or email us at securityunlocked at with topics you'd like to hear on a future episode. Until then, stay safe.Natalia Godyla:Stay secure. See for privacy and opt-out information.
Special Edition! We’ve been told for years how important passwords are, taught how to make them stronger and longer and better, and we frantically tear up our home or office when we can’t find that sticky note where we wrote them down. Life feels like it comes to a screeching halt when we’ve lost our passwords, but… what would life be like if we didn’t need them? Can your passwords truly become a thing of the past? Sounds a bit unnerving, but we can promise you, it’s always security first here at Microsoft.   On this special edition episode of the Security Unlocked podcast, hosts Nic Fillingham and Natalia Godyla explore the journey of becoming passwordless with Alex Weinert, Director of Identity Security at Microsoft, as he explains why your passwords don’t matter and how going passwordless can protect you from attackers.In This Episode, You Will Learn:    • The risks that are being mitigated through passwordless authentication • Where the challenges lie within using passwordless authentication • The functions of Windows Hello, Microsoft Authenticator and FIDO tokens • How ML is used in these technologies Some Questions We Ask:   • What does passwordless mean? • What are some common misconceptions or risks? • Where are customers on their journey to going passwordless? •What is the end goal for passwordless authentication? Resources:Alex’s Blog Post  Alex’s LinkedIn   Nic’s LinkedIn    Natalia’s LinkedIn    Microsoft Security Blog: Transcript(Full transcript can be found at Fillingham:Hello, and welcome to Security Unlocked, a new podcast from Microsoft, where we unlock insights from the latest in news and research from across Microsoft security engineering and operations teams. I am Nic Fillingham.Natalia Godyla:And I am Natalia Godyla. In each episode, we'll discuss the latest stories from Microsoft Security, deep dive into the newest threat intel, research and data science.Nic Fillingham:And profile some of the fascinating people working on artificial intelligence in Microsoft Security. If you enjoy the podcast, have a request for a topic you'd like covered or have some feedback on how we can make the podcast better ...Natalia Godyla:Please contact us at or via Microsoft Security on Twitter. We'd love to hear from you. Hi Nic, how's it going? Welcome to Episode 12 and welcome to three months of podcasting.Nic Fillingham:Yeah. Thanks Italia. This episode marks the, us passing the, the three-month mark, which is pretty cool, of Natalia and I being professional podcasters. I've actually put that on my LinkedIn profile now. So I think that makes it, uh, that makes it official. And I see you, we're obviously an audio only podcast, but as part of the recording, we have our cameras on. I can see Natalia that you appear to have embraced outward, which we, we talked about in the last episode. And you now appear to be in a small cave-like environment.Natalia Godyla:It does feel like a-Nic Fillingham:(laughs).Natalia Godyla:... cave-like environment. I can tell you that. I did transform my closet into my podcast studio. So it was a whole project this weekend. It's swanky, but I can tell you, there are some drawbacks. It is about 3,000 degrees in here.Nic Fillingham:(laughs).Natalia Godyla:I did not plan for that this podcast episode. So I'm, I'm dying a bit.Nic Fillingham:You're in the right place, though, if you decide like, "I'm not appropriately dressed for the temperature." You, you're actually in the perfect place to make that-Natalia Godyla:Yes, I, I mean-Nic Fillingham:... make that change.Natalia Godyla:... theoretically, yes. The other hazard of my current setup is getting locked in the closet, which has happened already. I did have to email for help.Nic Fillingham:(laughs). Who did you email?Natalia Godyla:So I emailed my partner who proceeded to Instagram, a picture of my email. It's just me in all capital letters asking him to get me out of the closet. So I'm glad that posting a picture to Instagram was of high priority in that circumstance.Nic Fillingham:Your partner was like literally feet away, right? Just, just drywall and framing away from you.Natalia Godyla:Yes, but I, I did an amazing job with my podcast studio. These blankets are intense.Nic Fillingham:Yeah. So like, were you banging on the window and the door and all that stuff? And he just couldn't hear you because the, the soundproofing was so phenomenal?Natalia Godyla:There was no knocking. Immediately, emails.Nic Fillingham:You were not, not even gonna to try, not even gonna try and knock. 'Cause I know, I know that I've done such a great job of deadening all sound. The only thing I can do is send a, an all caps email (laughing) subject.Natalia Godyla:The only option. This is all for our audience.Nic Fillingham:You know what? We had to, because our guests were coming on with better and better microphones, including the person you're gonna, you're gonna hear from today, Alex Weinert, who has a recording studio in his home basement. And he and I geeked out on bass guitars. But that, that wasn't the, the goal of the conversation. The goal of the conversation, um, was to talk about passwords.Nic Fillingham:And in fact, this conversation with Alex was so, was so awesome that we couldn't really edit it down. We've decided to do a special episode, which we haven't, we haven't done this before. Natalia, you're grieving away. Is there music coming through your headphones? What's going on?Natalia Godyla:No. I'm that interested in what you're saying, Nic.Nic Fillingham:(laughs).Natalia Godyla:I'm just grooving along with it.Nic Fillingham:L-, Natalia is literally like bopping away. I c-, I, she's bopping away to invisible music. Well, you, you take it from here. Tell us about the, uh-Natalia Godyla:(laughs).Nic Fillingham:You're obviously very excited. Tell us about the conversation we (laughing) had with Weinert.Natalia Godyla:Yeah. So we had a special episode with Alex, as you were saying. We talked about the future of passwords or perhaps the lack of future for passwords. So the inherent risks in continuing to use passwords is ... And some of the risks also with, uh, SMS, which I found really fascinating, the, the concept of it being out of bound, out-of-band and potentially then being intercepted.Natalia Godyla:Um, and then we just really dove into the reality of passwordless. What is the science behind building some of these password technologies? How real is it? How many customers are using it? So it was great to s-, get that substantive approach to passwordless, something that we keep hearing as a buzz term.Nic Fillingham:Yeah. This is a great episode to listen to after you, uh, get through Episode 8, which was with, uh, Maria Maria Puertas Calvo from the Identity team who talked about how that group utilizes artificial intelligence and machine learning. And then after we spoke with Maria, I think we might have been, we might have stopped recording at that point.Nic Fillingham:That Maria recommended that we then sort of move that conversation forward by getting on the phone or, or Teams as it is, uh, and chat with Alex to talk about passwords and the future, the history, the past, the, the good, the bad, the ugly of passwords. So it's a great conversation. We hope you enjoy it. On with the pod?Natalia Godyla:On with the pod.Nic Fillingham:Welcome to the Security Unlocked Podcast, Alex Weinert.Alex Weinert:Hey, how are you? Nice to be here.Nic Fillingham:Thank you so much for joining us, Alex, um, from your, uh, from your home recording studio, which we might, might touch on a little bit later. It looks, it looks pretty awesome. Alex, we normally ask people to first of all, sort of introduced themselves, and, and talk about their role. We will get to that, but I think I just want to sort of set the stage here. You are probably best known to our audience.Nic Fillingham:So let me know if you think it's fair to say you're best known to our, our audience as the, the author of the, All Your Passwords Belong to Us. Did I get that right? Or Your Passwords Don't Matter. You have some great blog posts, which really talk about the fact that passwords are bad. Don't use password. Is, is, is-Alex Weinert:Yeah, Your Password Doesn't Matter as a blog, that kind of took off. And then in my, in my, my non-blogging time, I'm the director of Identity Security for Microsoft.Nic Fillingham:Got it. And what does that look like? Like what, what does your team do? Sort of, what does, what does the day-to-day sort of look like for you, Alex? If there is-Alex Weinert:(laughs).Nic Fillingham:... if there is a, a standard day.Alex Weinert:Day-to-day. Um, I often joke that, um, I have a calendar that tells me what I'm, you know, I think I'm going to do on a given day. And then we have-Natalia Godyla:(laughs).Alex Weinert:... you know, various actors that, uh, change that agenda rapidly, uh, at times. First of all, you know, I think you, you spoke to Maria Puertas earlier. She's on the team. She's, uh, an amazing part of that group. And, and basically there are a set of functions that we do. We do internal security. So this is kind of thinking about, you know, how do we do secrets, um, management?Alex Weinert:And how do we set up our environment for dev ops, you know, security and, you know, pipeline security and operational security and all that kind of thing? And just making sure that the core of our identity system stays safe. And then, uh, we have an incident response team, which is sort of ... It would be nice to say the pointy end of the spear, but it's more like the windshield that catches the bugs, right?Alex Weinert:Like they, they deal with all the nasties that come in and, and try to hurt our customers or hurt Microsoft, uh, or customers via Microsoft. So that's another major function. And then what's cool is that this is where the sort of a flywheel starts, which is the things we learn from those investigations and those incidents go into Maria's team, right? And then Maria's team develops the refined, like data science that tells us, how prevalent is the pattern?Alex Weinert:How do we, you know, build detections into the product? How do we intercept those attacks and apply it in the product, so that we can keep them from ever hurting our customers? And then there's a set of teams that are kind of oriented around that signal that, that Maria's team produces. There's a signals intelligence team, which essentially packages that, so that customers can see it.Alex Weinert:There's the prevention team, which is basically about stopping fraud in the system and doing things in an automated way. So like one thing not a lot of people know is that we block, uh, something like 80 million attacks a day that customers never even know about, but we're able to see them. And, and so defending the system and defending customers from fraud, from account takeover attempts, that sort of thing.Alex Weinert:It's something that we do in an automated way on that team. So, um, the configuration by admins as to what credentials are allowed in the organization, and then the combination of that information with usage information and security information to decide, what's the right challenge sequence to show to a customer at a given time? That's, that's another team.Alex Weinert:And then finally, we have a team that is all about empowering end users. So we sort of jokingly call it the, like the karate school, right? Like it's, how do I teach my end users to defend themselves in a world where there's a lot of hostile activities? So the authenticator, which has the password manager feature.Alex Weinert:So that, that feature is part of that team, as well as things like self-service password reset and other, you know, the, the sign-ins logs that you can go look at and tell us whether you think the recent sign-in was fraudulent. And then all of that actually goes back into Maria's team and feeds that information to tune the algorithm.Alex Weinert:So when people, either administrators or users tell that they see something that we didn't notice, or that we got it wrong, that actually goes back in to make us more accurate. So that's kind of the flywheel, right? We go from incidents of bad things happening through data science and then ultimately out to the customer and to the end user and then right back into data science. And then, you know, by, by doing this, we're able to continuously train our systems.Nic Fillingham:Just for sort of scale, number of, of customers or, or number of sort of identities? I do-, I'm not sure what the right metric is here, but sort of we're talking in the hundreds of millions or are we in the billions category?Alex Weinert:Oh no. (laughs). No, like 40 billion log-in events a day, 170-Nic Fillingham:Wow.Alex Weinert:... terabytes of data, data generated per day. Yeah.Nic Fillingham:Wow, and, and, and the number of humans on the planet that are utilizing this, how's, ha-, how do we, how do we measure that? We measure that in the hundreds of millions as well?Alex Weinert:Mmm, billions.Nic Fillingham:In the billions.Alex Weinert:Yeah.Nic Fillingham:Wow. Okay. So these are bi-, pretty big numbers.Alex Weinert:Yeah.Natalia Godyla:(laughs).Alex Weinert:Yeah. (laughing). Relatively large numbers. Yeah.Nic Fillingham:Awesome. Thank you for that context there. So the, the, the, the topic that we sort of really wanted to start with here was, was passwordless. And, and we'll jump into that in just a sec, but I actually want to start with the fundamental of, you know, there's a lot of ... You know passwordless is, is, is sort of a newish term. It's sort of a buzz term. It's, it's being thrown around.Nic Fillingham:Can you define for us ... It may sound like a very simple question, but what is passwordless? What da-, what does it mean and what does it mean to us?Alex Weinert:Yeah. I mean conceptually, it is exactly what it sounds like, which is passwordless is when you authenticate yourself into a system without ever typing a password. The blog you mentioned earlier, you know, Your Password Doesn't Matter, it kind of goes into all the ways that, you know, short of using a password manager, it's basically impossible to have a, uh, a password that isn't in some way crackable.Alex Weinert:Um, so multi-factor authentication becomes a mandate, right? Like you have to have a second. If you're using a password basis, and you have to have something else. But the thing about it is that given how easy passwords are to crack, multi-factor auth reverts back to single factor auth pretty quickly in a world where your password gets guessed, right?Alex Weinert:So if your password gets guessed and you don't notice it, or you don't do anything about it, then you're now relying on a single factor, because the original factor is compromised, right? So the challenge we said is, you know, how do we get into a multi-factor authentication system where no password is present and actually try to not make that, you know, more challenging, but actually lower the usability bar? Like make it easier to use, right?Alex Weinert:And so what we looked at, uh, sort of in the initial way was Windows Hello, right? So in Windows Hello, you know, once you set up a device as your own, you can like literally just look at the camera and sign in, or you can touch the fingerprint reader and sign in. And the reason for that is that you have a biometric, right? Plus the device possession, and the device possession is hard mound.Alex Weinert:And so, you know, that model, like you think about that as FIDO is the same thing, except for it just takes ... It, it gives you more portability of the device you're using. So, you know, your, your FIDO tokens are, uh, you know, like on a USB form factor or in your phone PhoneFactor.Alex Weinert:And that allows you to then go from computer to computer and have that same, very strong authenticated experience on devices you haven't been on. And then the last one is the phone app, right? And the, the authenticator app is a way of doing passwordless, because we hard-bind into your phone. And then, again, there's some sort of, uh, secret. In the case of the phones, mostly it's Device Unlock.Alex Weinert:So it's, there's a, either pin or biometric unlock, right? So you're still doing two factors, but you're never having to interact with a password. So you don't forget your password. You don't write your password down. You don't pick a stupid password that ... Oh, I'm sorry. I, you don't pick a easily guessable password.Nic Fillingham:(laughs).Alex Weinert:Um, but I mean, seriously, password 1, 2, 3, come on. And by the way, the most common passwords in use are still things like 1, 2, 3, 4, 5, 6 and "I love you," and like, uh, things that are, you know, QWERTY I, uh, UIP, which is just about running your finger along the keyboard. It's like, so clearly people want less effort to go into their authentication rituals, right?Alex Weinert:So we're trying to figure out how to lower that effort bans, at the same time, make it stronger. The thing that is kind of unique, I think, in ... When we say passwordless right now in, in our authentication systems, we're talking about the authenticator application, Windows Hello and FIDO tokens. But I think we can extend that over time. FIDO gives us a nice framework, nice standards-based framework for extending that over time.Alex Weinert:There's an underlying thing that happens, which is really important. And I wrote about this in All Your Creds Are Belong To Us, which if you're old, like me, and you play old video games, you recognize the reference. Um, and, uh, and in All Your Credits Belo-, Are Belong To Us, we talked about something called verifier impersonation resistant. And that's sort of a heady technical term maybe.Alex Weinert:But what it basically means is that you can't put a machine in the middle of the ritual and trick the user, right? So one of the big problems we have with like tools like Modlishka is that Modlishka, um, does a pretty good job of exactly replicating the UI that the user's expecting to see. So the only thing that's protecting them in that case is that they ignore the ... If they ignore the cert warning, right?Alex Weinert:If they're not paying close attention to the URL they're going to, and that's really ... Unfortunately, most users aren't gonna to either get it, or they'll just literally bypass the warnings. So, um-Nic Fillingham:S-, sorry. What is, what is Modlishka? That's, uh, identity [crosstalk 00:14:07]?Alex Weinert:So Modlishka is a, is a red team ... It's like a pen testing tool.Nic Fillingham:All right, yes.Alex Weinert:And, and you can download it from GitHub, right? Like you can go search for it and download it. And what it does is it effectively, you point it at the server you're trying to intercept, request for. You're, so you're trying to machine in the middle, the request between the client and the, the legitimate server. And so this is actually ... We'll, we'll go super geeky for just a second.Alex Weinert:'Cause this is actually really an important aspect of passwordless that I think most people don't quite get. So basically what happens is when we have a, a situation where like you type in a password, and then you get, uh, an OTP code on your phone. The problem with that is that the communication is out-of-band, which means that the server is gonna say ...Alex Weinert:You know, they're gonna send or transmit a message to your phone and saying, "Hey, please approve this." Or, "Please," you know, "re-key this number." And then the user needs to key that number back in. If the user is tricked to going to, into a machine that is impersonating the identity provider, so if it's impersonating like Azure AD, that impersonalization is facilitated by a tool called Modlishka or other tools like it, that actually scrape all the UI code off of the original server and then replay it on their local server.Alex Weinert:So that's what Modlishka is doing is it's like replaying everything forward. So from a user perspective, this isn't like a hacky, lousy old version of the UI that doesn't look right. It's, it's going to look exactly right. It's going to behave in exactly the same way as the, as the code on the original server. So for a user interacting with that, they're like, "Well, this must be the real thing."Alex Weinert:The server will notice the anomaly. Like our server will notice the anomalies saying, "Hey, I don't think I've seen you on that machine before. So I'll challenge you for MFA." The problem is now the request for the MFA challenge is played forward to the user. And if we have an out-of-band authentication mechanism like SMS, the challenge now goes directly to that user's phone.Alex Weinert:Well, the user thinks they're interacting with us. So then they just key in the code that they got on their phone, right into the, the machine in the middle. The machine in the middle turns around and plays it back to us. We see that as an authentication pass, and then we would issue a token to that machine in the middle. And so that's how it's called OTP phishing. This is how like MFA bypass OTP phishing happens.Alex Weinert:So it's a slightly more sophisticated attack. The difference between that and password is that, uh, a pass should only attack is that if I have your password and there's no other protections, I can go anywhere I want and get new sessions. Whereas in this world, I have to trick you into giving me a session on one machine. And I've only got that session for as long as that token lasts, right?Alex Weinert:So it's a somewhat more limited attack, but it's still a very serious attack. And it's, it's a way to bypass e-, existing multifactor auth methods. So one of the really important things that's built into things like Windows Flow and, you know, FIDO and, and our passwordless methods is that we are looking at the, at the point where you issue the credential, at the point where we say, "Hey, that FIDO token can be used to sign into Azure Active Directory," for example, right?Alex Weinert:The credential is actually looking at the certificate of the machine that it's, it's taking a credential for. And built into the FIDO standard is this, this idea that you would never give the user an option to sign in to something that they haven't signed into before. So it won't ... The token itself will never even present the UI to the user to offer that token, because it'll say, "Nope, this is not a server that I've ever interacted with legitimately before. So I'm just not willing to give you a cred for it."Alex Weinert:So it defeats the machine in the middle of the attack, which is a really important and cool thing that it does. So that thing where you look at the credentials of the service that's asking for the credentials, that's called verifier impersonalization resistance. So that was super nerdy, but it's a really important aspect of this thing, which is that we have a cryptographic relationship between the token that's being used to sign in and the service that it's being used to sign into.Alex Weinert:That's two-way. The trust is both ways. So the, the token has to trust the service too. So if you try to impersonate that service as a machine in the middle, your host, like you're not going to ... It's not going to work. And that's a really cool thing about passwordless. So not only is it, you know, you're not going to write down the password, you're not going to choose to use the guest password, all the other issues with passwords.Alex Weinert:It also bypasses many of the vulnerabilities of existing multifactor auth that is out-of-band in nature.Natalia Godyla:So you've outlined a number of risks that we're trying to mitigate through passwordless. Uh, just thinking about it from the other side, wh-, what are the risks that are still inherent and passwordless? What are, what are some common misconceptions on what it can solve? What should people be continuously aware of even after they've implemented passwordless, other identity technologies that need to be paired with it?Alex Weinert:That's a great question. Um, I think that for those of us who've been around the identity industry and the security side for a long time, uh, the, the thing that we probably worry the most about is, uh, what happened around smart cards, right? And so smart cards ended up being a very secure mechanism that was very niche-y in nature. And the reason for that was that there were serious usability issues and, and manageability issues at the, at the organizational level.Alex Weinert:So for example, if you lose a smart card, you know, you leave your smart card at home, you come to work without it, how do you go to that person, authenticate for the day? And it turns out you need a way to manufacture a new smart card. And that is, uh, an expensive process. And you need to physically get somebody down to a desk and, and issue and all that sort of thing.Alex Weinert:So the form factor, specifically the fact that we had to embed credentials using specialized hardware was kind of a big deal in, in those days. And so, as we went into the new generation of passwordless technologies, we wanted to get the security benefits of, of the, sort of the old PIV and smart card model. But we wanted to do it in a way that we could get great usability as well.Alex Weinert:And so the major things that I think we worry about are actually on that usability spectrum. Like if I have a really strong credential ... Le-, let's, let's first back up. Let's talk about passwords. How many places will give you a password reset based on knowing your mother's maiden name or your last address?Natalia Godyla:(laughs).Alex Weinert:Right? Why would we have-Natalia Godyla:Sounds familiar.Alex Weinert:... such a weak mechanism to, to recover a password? And the answer is because passwords are so intrinsically weak that a weak mechanism in some senses is like a rational response. But when we get to a place where we have like a FIDO token, which is a cryptographically, you know, like ha-, hardware-based cryptography, and it's awesome, right? Do we still want to use your mother's maiden name as a way to recover the credential?Alex Weinert:And so recovery becomes one of the brass ring things that we need to go make sure we get right. So issuance recovery, all the things that are about getting you started. Now, for organizations that can use phones, like this is a great way to go for a lot of organizations, if you're allowed to use your mobile phone in the organization. So you can use the authenticator app.Alex Weinert:We've done a ton of work to have essentially the ability to generate a temporary credential issuance code as a help desk, and then have somebody simply point their phone to the screen and get their new credential. And so some of that, we've like massively lowered the cost and the effort involved for an organization to manage these things.Alex Weinert:But then there's organizations where you're not allowed to use a phone, right? They're either, because you're in a secure environment where phones aren't allowed, you're on a retail floor, or there are union or governmental regulations that prevent requiring, or allowing a customer ... Um, I'm sorry, a user to use their personal devices. Right?Alex Weinert:So then you have this whole issue around, okay, so now you've got hardware. And, so what happens if somebody has a two-hour commute to work, gets there and realizes that they left their, you know, FIDO token on their other key chain, right? Like what happens if, you know, you're borrowing the car or your car is in the shop, whatever?Alex Weinert:So the thing that is of concern when you go into these really strong credentials is that you have to have a pair-wise, really strong, you know, lost, forgot recovery and issuance flow. Like we've had the basic login to windows with a FIDO token working for, I think, a couple of years now, right? Like that's not where the energy is going right now. The energy is going in the usability piece.Alex Weinert:Like how do I get to a place where you can go order a FIDO token from your favorite online retailer, have it show up in your house, you know, via Speed Delivery? Right? So one of the scenarios we talk about is if I'm traveling and I get robbed, right? Like, and I need to get into my machine, what do I do? Right? So I can order one of these things, retail off the shelf.Alex Weinert:I can interact remotely with my help desk. And then I can actually reprovision the strong credential from right there on my, my laptop, you know, in my hotel room, right? Like that ... And I realize this is, you know, the pre-COVID version of this, but it's in fact more relevant now. I've hired, I think, you know, something like 10 people onto my team since March. Not one of those people has had physical contact with anyone from the corporation, and they're all doing strong credentialing. Right?Alex Weinert:And so that, that bootstrapping process is really important to get right, especially now. That's where the real challenges are. I don't think that there's a significant argument to be made for, for the security side of this at all. Like the security here is as good as it gets, short of ... I mean, we're certainly just as good as it gets, right?Alex Weinert:You, you could add other rituals, like manager approvals and that sort of thing. Well, you can do that now. From a credentialing perspective, you don't get much better than a cryptographically strong device where the crypto's being done in hardware and you're validating everything all the way down the chain. The people that worked on FIDO2 did a good job, right? They, they nailed the security promise.Alex Weinert:What we're trying to nail now as the usability promise. And even that on the mainstream line isn't that hard, but when you get into the, "Oops, I," you know, "I washed my FIDO token in the laundry today," right? Like that becomes more of a problem. And so how do you reestablish trust? That's a place where we're putting a lot of investment. And I think that that will be the make or break for, for strong credentials.Alex Weinert:The thing about passwords that, as much as I would like to see them eradicated from usage, the thing about them is, you know, there's essentially an infinite key space. They're super easy to reissue. The user can self-reissue. Like there's a bunch of ease of use stuff around passwords, until you forget that, and that's a whole different problem.Alex Weinert:When you, once you get to a really strong credential, you have to kind of match up the ease of use piece. And that's a big investment.Natalia Godyla:So where are customers on their journey to passwordless? We're at a point where we're improving what we already have. And so, like you said, we're focusing on usability. Are our customers actively using these methodologies? Is there one that is preferred over others? What does that look like for people?Alex Weinert:In broad strokes, adoption of Windows Hello is terrific. Like we have many, many, many customers that their primary sign-in mechanism every single day, as you open your laptop and you get to work. And there's a cryptographically strong handshake happening there, but you don't as a user, think very much about it. You can use a pin, face print, thumbprint. I use a pin ...Alex Weinert:Confession time, uh, because I'm on this crazy deck here, and my, all my scanner, all my actual computing hardware is way over on the side. So the pin is an easy way to do it from the keyboard. But if you were using a, a, like a face scanner, which is built into most laptops, the camera will work in same way that you would look at your phone to unlock it. Then you're just signed in and you don't think about it. And that's a really great user experience.Alex Weinert:And that's actually the experience you're used to on your mobile devices. It's the experience customers are used to on their, on their Windows devices. Then the next place that we see really good traction in, you know, here, it's tens of millions is in the authenticator app, right? So the authenticator app is a very popular option for people to use. It's on the phone. So you want to sign in.Alex Weinert:You gotta, you know ... Thing flashes on your phone, it says, "Please approve." And then you push the number, you know, that matches the screen. And that I think has driven a lot of adoption of the authenticator app. So the authenticator app is the second most popular. And then with FIDO, I'd say people are dipping their toes in the water. Like organizations are getting serious.Alex Weinert:people that wear a lot of tinfoil hats like me, you know, the overall Net/Wall or mission full hats, right? Um, are, are deep into the FIDO experience. And so I sign in every day, uh, using FIDO because I, I know the, you know, the security promise behind it is just outstanding. So m-, my personal accounts, I don't have passwords that I know on any of my personal accounts. I intentionally put random, random strings into all of my password fields as-, and then destroy the strings, so I don't have a copy. All of my sign-ins every single day are passwordless.Natalia Godyla:So you mentioned that, uh, the scenario in which you find out that there has been something suspicious in your account and you respond to the request. But ultimately there's something in the technology identifying something as suspicious. How does that work? Are we using machine learning for that use case? Uh, uh, how do we use it across all of the technologies that you've described?Alex Weinert:Yeah. So back in the beginning of my journey with this team in, I guess it was 2013, we were struggling with the fact that we would, um, go through this process where we would figure out a new attacker signal and we would update our algorithms. And that would take a certain amount of time. And then we would test and we would package and we would deploy to servers all over the world and the fix would go live and the attackers would be disrupted for about a day.Alex Weinert:And then they would adopt to our new algorithms and we had to start over. So we were on like a sort of six-week cycle, you know, to get changes made. And then they were on a sort of a two-day cycle to respond to the changes. And so we were on, you know, what, I think a lot of people who have a long background in defender technology know, which is that it can feel like a treadmill.Alex Weinert:Like you, you take a step, that you take a step and then you're right back where you started. And so we made a bet on adaptive defenses, on adaptive technology for defenses. And that was a really hard bet. I mean, it diverted a bunch of resources and stressed a lot of people out and it went on ... You know, we had a lot of false starts. We've talked to other f-, friends in the industry who, you know, started and abandoned their efforts in this area, because it, it can be frustrating.Alex Weinert:But we got to a place where we could beat our static heuristic algorithms with our machine learning algorithms. And at the time, we looked at like 30 different features. A feature is just an aspect of a log-in, right? Like some ... It could be your IP address. It could be your browser, you know, your agent string, whatever, but we'd look at these things.Alex Weinert:And we looked at like 30 and we would say, "All right, given this combination of factors, what's the probability that this thing is going to be a good log-in or a bad log-in?" When you get into data science, you, you're working with two things. There's precision, which is the number of times, if I say it's bad, how often is it r-, is it really bad? And precision is really important, because it's, it gets into how many times do you artificially challenge a user?Alex Weinert:And that results in user friction and like bad experiences and help desk calls and costs. And people will turn off security technology that gets in their way. And this is an unfortunate truth, right? Like if you put technology in front of your users, that frustrates them. Even though it's the, doing the right thing from a security perspective, the organization will turn it off, because productivity is the higher order bread for every organization.Alex Weinert:And so every CSO knows this and has to live with a sort of balance, right? So one of the things that we have to do as security professionals is we have to put experiences in front of people that actually enhance their experience to the extent possible, or at least minimally disruptive. So precision is the thing that we look at for that when we match the precision of our then best algorithm, which was at around 17%.Alex Weinert:Which means that eight out of 10, roughly eight out of 10 challenges that went to users were unnecessary, right? We were, you're throwing MFA challenges that users are blocking them incorrectly, eight out of 10 times. When we match that with our la-, machine learning stuff, when the machine learning got as smart as our current static algorithms, we started blending the two together and then the machine kept on getting better and better and better.Alex Weinert:And over the close of about four or five years, it got up to, north of 85% precision. On the enterprise side, you're given some flexibility. You can say, essentially, "Hey, I'm more risk sensitive," or "I'm less risk sensitive." And so you can tune that precision. But the other side of the equation that moves is recall. Right? And so recall is how much of the bad traffic are you actually catching? Right?Alex Weinert:So I can get precision to a hundred percent if I simply never challenge, right? If I basically never ever challenge, then I will never bother a good user. And I can say, "Yeah, yeah, yeah, I have nothing wrong," but the problem is I'm also catching no attackers. And in that world, um, I want the best possible recall. Or I could simply challenge everyone, and I can get a hundred percent recall, right? I can bother every good user and everybody. I'll get all the bad users.Alex Weinert:So you, the, the thing that's super tricky in this space is turning that dial to the right place. And so machine learning has done huge amounts for us in that space. So we just recently had an algorithm that was static. And when I say static, I mean that is not machine learning, right? Is traditional heuristic algorithm, that detected a, a, an attack called password spray.Alex Weinert:And our password spray algorithm was about 98% precise, which means that, like, if we said it was a bad user, it was a bad user, you know. 98% probability. We were able to double the recall of that by applying machine learning to it. Like we took the supervised machine learning technology and applied it. And after a brief training period, we released it and we hit, doubled the recall without moving precision at all. Right?Alex Weinert:So that's fantastic. Right? Our precision stayed high and we doubled the amount of bad actors we're, we're catching. And one of the things about recalls, you never know the, the total number, right? 'Cause you don't know what you don't know, unless you're in, in like a thing where you can ...Alex Weinert:There are machine learning environments that you'll see if you go to like conferences, which are all like, "Okay, I had temperatures of cats and temperatures of dogs. And my machine learning algorithm is training." And in a world where you're like in a constrained dataset fine, but attacker's whole job is to be invisible. Their whole job is to, to defeat the machine learning system.Alex Weinert:So when we look at a r-, like doubling of recall, that's a significant step to do that without moving precision at all. And, uh, the team was able to do that. That particular system looks at over 200 aspects of every log-in. And then you're, it uses the machine learning algorithms to, to figure that out. But the most important thing about it is that it will, without our investment, without significant investment, continue to get better.Alex Weinert:And of all the things machine learning did for the team and for the defenses of customers, I think the most important is that it freed up innovation cycles. Like the humans were able to go back to really innovating on, how do we find new attacks? How do we defeat these attackers, w-, while the system continues to do the things that we used to do manually? Which is, "Oh, look, a new parameter. Let's tweak the parameter and propagate it." That's now happening for us automatically. So we can go off and invest in innovation.Nic Fillingham:I just want to maybe get some clarity on, on one little piece there. So I use the authenticator app myself. Obviously, you know, I'm a Microsoft employee, so I, I have to use that for my, my job, but I also use it personally for, for personal services. Every now and then, I do get a ping on the authenticator app that doesn't appear to be from something that I've initiated. It's rare, but it does happen.Nic Fillingham:Can you ... This is a slight digression here, but like what's, what's happening there? Is it always a sort of a malicious act happening on the other side of the, of the coin and the fact that I'm ignoring them, obviously, because I don't initiate it? Is that good? Am I doing the right thing? And is that actually helping the model get better? What, w-, what, what happens in those sort of, I guess, false positives? Is that what it's called?Alex Weinert:Yeah. Well, so that's not necessarily a false positive. I mean, I'm not sure I would call it a false positive. So let me tell you about the, the things that will cause that. The two things that will cause that are an attacker has tried to log in. If you're getting a, you know, the, the three codes presented thing, and, and you have a account that's set up for passwordless, and they might've just typed in your username and they're trying to sign in, obviously you should never hit approve on a request that you don't know where it came from. Right?Nic Fillingham:Right, yes.Alex Weinert:I'd like to be very clear. The other possibility is that you have legacy software that is like, you've, you've left a client running somewhere. And this was the cause for a lot of, um, multifactor authentication and things that don't get answered. Because we have blocks in the system, like you have to complete your phone number entry or whatever, that, that require that before you take that next step.Alex Weinert:But if you have software that is like, "I'm gonna try to log in," and that trips a, a multi-factor authentication challenge, then that can be the other thing that happens sometimes. That's pro-, the primary two. Um, we're, we're doing a bunch of work right now and I, I won't get super specific, but I'll say we're doing a bunch of work to make it hard or nearly impossible to approve a malicious attempt at logging in.Alex Weinert:And so, you know, we have ... The wonderful thing about the authenticator app is in some sense, like our systems, we can adopt it very rapidly, and we can adapt the UX for it very rapidly. So the team's putting a bunch of energy right now into this question of, how do we tune the authenticator, so that users don't do accidental approvals and they don't, you know, respond to those, those kinds of challenges?Alex Weinert:But yeah, the majority of those will be caused by either an attacker who has your username and password, and is tripping the, you know, the last step of the authentication or, uh, an old application that doesn't know that it's triggering MFA.Nic Fillingham:Got it. And so me, me ignoring that, though, am I actually helping? Is there some other step that I should take to say like, "Oh, I don't think I actually requested this?" Like, how do I actually help the machine learning models get better to reduce the times that, that I would see those challenges when I don't request them?Alex Weinert:You can review in, uh, My Sign-ins. You can review that either on the web or on your phone. And then you can indicate that a given log-in request was, or wasn't. You know, they can also help you understand whether your, uh, password is compromised. So for example, if you see someone who got through the password challenge, but got stopped at your MFA challenge and it's coming from a country you've never been to and on a device you would never use, right?Alex Weinert:You click, "This wasn't me," and then we will actually step you step by step, how to re-secure your account. And so this is an important part of our security apparatuses to, you know, get the user involved, and we can walk them through re securing their accounts at that point. So that's kind of the best thing to do. If you're getting challenges, you're not expecting, go look at your sign-in logs and, and then react, you know, if you see something out of, out of whack.Nic Fillingham:That's great advice. Thank you. And I want to touch on one, one other thing that you said. So is the end goal for passwordless that there are no passwords anywhere, or is it simply that a password may exist, but the end user basically never enters it? Is that, is the end goal that on my, my identity, my account, my user entity-Alex Weinert:No.Nic Fillingham:... there is no actual password in any shape or form associated with that, and instead it is things like a FIDO key or some other authentication mechanism? Or is it simply that the password does exist, the user just never, never has to enter it?Alex Weinert:Yeah. Well, so we should be clear with that. I think th-, there are, you know, there are systems that still run FORTRAN. There are systems that still run COBOL. Like-Nic Fillingham:(laughs).Alex Weinert:... VAX assembly systems are still out there. Like you're going to have, you're going to have a long tail of technology that is highly coupled to passwords for a very long time. And, and so some passwords will still exist in the environment. Our, our goal is, uh, as we get users into their sort of daily ritual, that that does not involve a password.Alex Weinert:If you have a password you don't know that is also cryptographically strong, so it's, you know, it's completely, what's called entropic, which means that it's a string that doesn't have any patterns in it at all and it's totally random, then that, and not having a password at all are about the same thing. Right? Which is why I've essentially rendered my accounts passwordless without actually like having a system underneath it that deletes that thing from the environment.Alex Weinert:So yes, the goal, I think long-term ... And I, um, say two things here. First of all, the goal here long-term is absolutely the eradication of what is the weakest possible link in s-, in cybersecurity. And we have moved on from the world where I might want to do the, you know, Tom Hanks, Meg Ryan, you know. You've got mail thing. Like that, that's one bar. And now we're talking about like national infrastructure and like global economies and healthcare, and, you know, like lives on the line who are behind these passwords. Right?Alex Weinert:So we, we have to realize that we've kind of shifted our, our security mandate in a pretty substantial way when we're betting the world's infrastructure on the integrity of logins. And so to say it's okay to have like QWERTY I, uh, UIOP as your password, if your password is guarding something like whether the trains run in Europe or whether, you know, lights come on in Minnesota in the winter, right?Alex Weinert:Whether the heaters can come on, like, these are bigger deals than somebody like intercepting a personal mail from the days of bulletin boards. Right? So I think we have to, we have to say, we, we have a mandate to get past the password. So I believe very strongly that yes, our goal here is to find ways that are, that, that are in line with our expectations, for security, for the kinds of systems we're securing now.Alex Weinert:The second thing I will say is that, okay, so it's a long tail. The mitigation for passwords is MFA, right? The mitigation is multifactor auth. And as much as I would say your best bet for multi-factor auth today is probably the, the ma-, the authenticator app where you're doing cryptographic communications and, you know, you have all sorts of other hardening, any multi-factor auth at all of any kind dramatically reduces your risk of compromise, like really dramatically, like more than 99.9%.Alex Weinert:So when we go look at the body of compromised logins that we have, we'd say, "All right, here's all log-ins that we definitively said these were bad, right? These were cases where an attacker got in," only one in 10,000 of those will be a non or will be an MFA'd account. Okay? So that, that's how like radical this is. So if I go look at all my compromised accounts, all the compromise that happens in the system, only one in 10,000 of those will have MFA.Nic Fillingham:And therefore, if you have MFA-enabled, you are protecting yourself from ...Alex Weinert:Vastly, vastly. Right? Like, and even targeted accounts, targeted attacks very often are defeated by conventional MFA. Because as much as we would rather ... Like when we, if you look at something like the radio intercept stuff I write about in the Hang Up The Phone blog, we should be clear that like that radio intercept stuff is, um, it requires proximity in most cases. SS7 doesn't, but the other ones do.Alex Weinert:So if I want to intercept your cell communications, I need to get close enough to you to do it. So I have to get, you know, physically close. Well, a lot of attacks are taking place from around the world. Right? And so it's, it's hard to get close to somebody. So once I have MFA, that requires proximity, I'm going to like, "Meh, I'll give it up." You know? So as long as you're, you're not blind approving things, um, and your phone provider isn't giving away your account, right? Which is an issue. You are probably okay, you know.Alex Weinert:And you were certainly a whole lot better off in not using MFA at all. So I think we have to think of this as tiers. Like password-only is the worst. Password p-, plus MFA is, with, with phones is the next. It's much, much, much better. Right? And then we would say password plus MFA with non-phone mechanisms is the one after that. And then we would go from there to say, "Okay, let's go passwordless with, you know, pho-, with the phone authenticator.Alex Weinert:And to be clear, I'm talking about an application, not the, not SMS, right? Or Windows Hello or FIDO. Like now you're into the brass ring neighborhood. You're like, you're doing as good as you can possibly do.Natalia Godyla:Understandably, Alex, we still have a lot of work with securing the institutions and enterprises. As you said, uh, organizations like utilities still need to adopt passwordless, but what's next after passwordless? Let's say everyone goes passwordless. What is the remit for your team? What are you going to focus on?Alex Weinert:On my tie, uh-Natalia Godyla:(laughs).Alex Weinert:(laughs).Nic Fillingham:More, more bass guitars. More, uh, more music recording?Alex Weinert:Yeah. More bass guitars in a warmer climate. Yeah. The, um ... No, I think ... So there are a couple of inevitable places that attackers will be forced to move, um, once, once we get to secure authentication for users. So if everyone was using ... Let's be very clear. If everyone was using MFA, we would see a big surge in, uh, MFA phishing. Right? We'd see more, uh, Modlishka style attacks, like I talked about before.Alex Weinert:Um, if we get everybody to FIDO and we say, "Okay, now it's impossible to forge a token," then what we have to look at is token theft, which is where an attacker is trying to get into your box as a system, as system memory, lift the token out and take it somewhere else. Um, so for that reason, we're investing very heavily in proof of possession token binding, and, uh, trying to make that an impossible thing to do.Alex Weinert:So I think that the key things here, as we, as we think forward become things that are less user-centric in nature. Like we ha-, once we get users using the right kind of credentials, then we shift into the underlying systems to really harden against, you know, malware attacks, token theft attacks, um, and other things that are very nuanced and, and require a conversation between all the components to get right.Natalia Godyla:Thank you. Thank you for that look-ahead and for joining us on the podcast today, Alex.Alex Weinert:Thanks a lot. It was really fun.Nic Fillingham:I'm gonna go change my password from QWERTYUIOP on my Hotmail account. That's probably out of date now.Alex Weinert:Right. And add MFA while you're on it. Well, your, your Hotmail account has MFA, but (laughs).Nic Fillingham:Perfect. Thanks Alex. We'd love to see you again on a future episode of Security Unlocked.Alex Weinert:All right. And we'll have to talk bases again some other time.Nic Fillingham:Definitely. Thank you.Alex Weinert:(laughing), all right, see you.Natalia Godyla:Well, we had a great time unlocking insights into security from research to artificial intelligence. Keep an eye out for our next episode.Nic Fillingham:And don't forget to tweet us, @msftsecurity, or email us at with topics you'd like to hear on a future episode. Until then, stay safe ...Natalia Godyla:Stay secure. See for privacy and opt-out information.
How do we ensure firmware integrity and security? Join hosts Nic Fillingham and Natalia Godyla and guest Nazmus Sakib, a Principal Lead Program Manager at Microsoft, to dive deeper and assess the complexities and challenges that come along with securing firmware - bootstraps and all!  Megamind Bhavna Soman, a Senior Security Research Lead, joins us later in the show and we learn about her journey in optimizing AI and ML to improve efficiency in security and give the humans a break.  In This Episode, You Will Learn:  • How Microsoft ensures firmware integrity and security • How firmware is making it harder for attackers• Where AI and ML will take threat intelligence in the near future Some Questions We Ask: • What is firmware? • Do we know where firmware attacks begin? • What does the threat landscape look like for firmware? • What part of ML should be automated better so that humans can shift to other tasks?   Resources   Microsoft Digital Defense Report:    Nazmus’s LinkedIn    Bhavna’s LinkedIn  Nic’s LinkedIn    Natalia’s LinkedIn    Microsoft Security Blog:  Transcript(Full transcript can be found at Fillingham:Hello, and welcome to Security Unlocked, a new podcast from Microsoft where we unlock insights from the latest in news and research from across Microsoft security, engineering, and operations teams. I'm Nic Fillingham.Natalia Godyla:And I'm Natalia Godyla. In each episode, we'll discuss the latest stories from Microsoft Security, deep dive into the newest threat intel, research, and data science.Nic Fillingham:And profile some of the fascinating people working on artificial intelligence in Microsoft Security. If you enjoy the podcast, have a request for a topic you'd like covered, or have some feedback on how we can make the podcast better-Natalia Godyla:Please contact us at or via Microsoft Security on Twitter. We'd love to hear from you. Natalia Godyla:Welcome to the latest episode of Security Unlocked. Welcome back to civilization, Nic. I'm hearing that Seattle had a pretty bad windstorm. Glad you're okay.Nic Fillingham:Thank you. Yes we did. We were out of power and internet for best part of two days. That was fun. But yes, we're back online. We have power. We have internet. We're back in the 21st century. How about you, Natalia? Any insane weather events up in the northeast? You guys get ice storms and cats and dogs and locusts falling from the sky, don't you?Natalia Godyla:None this weekend though. I did almost freeze going camping and I had a close call with an attack over the weekend.Nic Fillingham:Oh my gosh, that sounds crazy. What happened?Natalia Godyla:I mean, it happened in Outward. I feel like I probably should have started with that. But Outward, the game.Nic Fillingham:Oh, okay. Phew. I feel like you would have mentioned that to me in advance of recording this podcast had you actually been attacked in real life. What's this game? What's the game you're playing?Natalia Godyla:It's an RPG game where you try to quest through this... Gosh, I don't remember a single name of any of the locations, the cities, or the mountains. I'm not paying attention. I'm really focused on the battles that you have to fight.Nic Fillingham:What are you battling? Can you give something away or is it a spoiler? Is it humans? Is it animals? Is it zombies? Is it aliens?Natalia Godyla:It's a mix. There are bandit camps and then there are troglodyte caves. I think I've taken on a whole lot of the troglodytes at this point though. So I don't know if they're still in existence.Nic Fillingham:Let's take 30 seconds to look up Outward. You said troglodyte, and I really feel like troglodyte is an established word that means something. Oh, okay. So troglodyte is from the Greek, troglodytae, which literally means cave goers. Is that right? Do they live in caves?Natalia Godyla:They do live in caves.Nic Fillingham:Oh, there you go. Okay.Natalia Godyla:This game must have done its research.Nic Fillingham:They're cave goers, but they're also your enemies. Is that right?Natalia Godyla:Yes, but I guess in theory, I brought it upon myself. I mean, I kind of wanted to loot the cave.Nic Fillingham:So you actually went into their territory and were like, "I'm going to smash this jar and get this green jewel out of it." And they were like, "Hey."Natalia Godyla:Yeah. I mean, that's a moral gray area because they saw me and immediately attacked but it was their cave.Nic Fillingham:So you're the bad guy. Nice. All right. We're going to play this. We're going to play Outward. Wonder if we can get all of the Security Unlocked peeps into a single game. That'd be fun.Natalia Godyla:Oh, yes. And I think with that, we can intro our guests. Yeah, there's no connection point here.Nic Fillingham:Speaking of cave-Natalia Godyla:Looting.Nic Fillingham:Looting? No.Natalia Godyla:How do you stop looting from happening?Nic Fillingham:Oh, got it. I got it. If only those troglodytes had better security, people like Natalia Godyla wouldn't just come wandering in to ransack the place looking for leather and iron ore to craft rudimentary weapons. Speaking of better security, today on Security Unlocked, we talk with Nazmus Sakib who is going to spend a bit of time talking to us about firmware and the challenges associated with ensuring firmware integrity and the integrity of device security all up starting with firmware. This is going to be the first of three conversations that we'll have over a number of episodes where we better understand the security of devices from the firmware up. And then after that segment, Natalia, who do we speak with?Natalia Godyla:After that, we speak with Bhavna Soman who is a senior security research lead at Microsoft. And she shares how she got into security, which was really a role that she played in her family. She was the de facto support for network and security issues like antivirus. And as she continued in that role, she got more and more curious and tried to understand what technicians were changing or why something might be affecting her computer. And that role and responsibility just made her that much more interested in the security space and eventually led her here to Microsoft where she works on understanding and getting insights from the data that we have in order to better inform our defender products. Onto the podcast.Nic Fillingham:Onto the pod.Nic Fillingham:Welcome to the Security Unlocked Podcast, Nazmus Sakib or Sakib as we'll call you in this podcast. Thank you so much for joining us.Nazmus Sakib:Thanks, Nic. Thanks Natalia for having me. It's a pleasure to be on here.Nic Fillingham:So two things. We love to start up these deep dives with an introduction. Would you mind explaining? I introduced you as Nazmus Sakib, which is your name. We're going to call you Sakib. Just anything you want to sort of say about that, but also, what do you do at Microsoft? Tell us about your role, the team you're in, the team's mission? What is your day-to-day like?Nazmus Sakib:Yeah. I'm Nazmus Sakib. I go by Sakib. It's usually a sign on the team that you've met me where I get to clarify that growing up, everyone just called me by my last name. I'm originally from Bangladesh and Sakib is just more common as a first name in Bangladesh, which is what most people... My family ended up calling me. There's a famous cricketer by the name of Shakib Al Hasan who some listeners may be familiar with, but this is my first foray into fame.Nic Fillingham:I am familiar with famous Bangladeshi cricketers. Thank you very much.Nazmus Sakib:He's finally back after an unfortunate ban, but I think it's great to have him back on the team. Super excited for the prospects of the Tigers.Nic Fillingham:Do you play cricket? We're going to do this. We're going to take the little party.Nazmus Sakib:Yeah. Let's go down fully on that rabbit hole. So I played a lot when I was younger. I've been in America mostly since 2008, is when I first came for college. But prior to that, like most I think kids in Bangladesh, we play cricket. And usually, I grew up in Dhaka, which is the capital. So it was all improvised for the longest time. We had a little space on our roof. So it was like this flat essentially. And so it was probably about maybe 10 feet by 10 feet or not even. And so me and my cousins be a team of like two or three kids and we'd split it up. Someone would bat, someone would ball. You'd make up the rules in terms of how the runs would work. And same thing with if you find a little space in a back alley, or in any small sort of field or space that you'd get, you'd find a way to make it a cricket field. So good memories from back there. So it was kind of informal, but a lot of fun, especially now that the years have sort of gone on and I'm in a much different place where you just don't do that. It's pretty cool memories.Nic Fillingham:Bring us back to your role here at Microsoft and sort of what you do. Can we think of a good cricketing segue? Is there any famous cricketers that have moved into the cybersecurity field? What's a hard left turn?Nazmus Sakib:I think Satya is obviously-Nic Fillingham:Oh, yes, Satya loves cricket. He's a big cricket fan.Nazmus Sakib:Satya loves cricket, yeah. So I guess he's the most famous former cricketer turned tech luminary that I can think of.Natalia Godyla:10 points for the connection there.Nazmus Sakib:So yes. It is a well worn path, cricket to Microsoft. And I'm just one more traveler on that road. But my day-to-day, I've been at Microsoft for a little over eight years now, actually right out of college. I work as a PM in one of the many security teams at Microsoft. My team currently is in the Azure Edge and platform team. Our team is responsible for the operating systems that we ship as part of Microsoft, and also that operating systems that our customers use on platforms like Azure. So our team has been responsible for building the security that goes into Windows for a long time. Been a part of that team since I started at Microsoft.Nazmus Sakib:And then with the way to serve our customers on Azure, we want to meet them where they're at. And we have a lot of Linux customers on Azure as well. And so increasingly, our team is not just doing Windows work. We're also investing in Linux security technologies to help ensure that if you're a customer coming into Microsoft, if you're using Azure, whether it's on Windows or Linux, really bringing that platform, that operating systems' expertise to help secure whatever it is that you're you're trying to do. Nic Fillingham:Awesome, thank you. I'm really excited for this conversation we're about to have. It's going to be one of sort of three. I won't call them introductory, but it's certainly a little trinity of conversations over the next few months where we're going to talk about firmware. We're going to talk about firmware integrity, the challenges of that, and how you go about ensuring and securing firmware integrity. We're going to follow that up in a future episode talking about the Microsoft Pluton announcement. I'm sure that'll come up at some point in our conversation today. You're joining us today, Sakib, to help us sort of come back to basics a little bit. Can you help orient us in this world of BIOS, UEFI firmware, all the various sort of synonyms for this stuff? We're going to talk about firmware. Let's talk about what is firmware. Let's talk about these acronyms. If you would, just sort of re-educate us so we can start the conversation.Nazmus Sakib:Right. So the easy way to think about firmware is it's the first piece of code that runs on your hardware, right? So it's easy to sort of visualize that when you have a device, it's a desktop, or a PC, or a phone, any kind of computing device, you have the actual hardware, right? You've got the CPU, the motherboard, the power button that you use to turn the whole thing on, you have the hardware. The firmware is really essentially software that's typically baked in to the hardware. So it ships typically as part of the hardware. There's usually some read-only memory chip that's dedicated to storing that firmware just so that when a customer hits the power on button, the hardware knows how to turn everything on essentially. It's the firmware, that piece of software that actually goes and coordinates how devices are being made available to all the other things that run after the firmware, which is the operating system, and then the applications that you use on top of the OS.Nazmus Sakib:So if you were to think about from the point that you turn on a device to the point where you're using an application, whether it's your browser, whether it's Teams or Zoom because it's COVID, usually a very simple workflow for that is you're turning on the hardware. The firmware is the first piece of software that runs on the hardware platform. It bootstraps the operating system. So it could be Windows, it could be Linux. And then after that, once you have the operating system running, you can run applications like your browser, Teams, Zoom on top of that operating system platform. Nazmus Sakib:So the second part of your question, what is BIOS or UEFI? They're essentially flavors of firmware. BIOS has been around for the longest time, I think, in many ways with the history of the IBM PC. The BIOS was what you'd call essentially the firmware that ran on an IBM PC platform. A few years ago now, I think, essentially, the industry got together to revamp the firmware standards. So it's both a specification and an implementation of that specification. So UEFI, you can think about it as the modern BIOS, but because historically, people called firmware BIOS for the longest time, they're almost essentially synonyms. But typically, BIOS and UEFI both refer to the firmware that runs on any particular platform. And in general, they're perhaps used synonymously if we're speaking loosely. But most modern systems today use some implementation of the UEFI specification as the platform firmware.Natalia Godyla:Can you provide some security context around firmware? What does the threat landscape look like for BIOS or the broader term firmware? What's been the history of attacks? What's more or less prevalent for firmwares compared to applications that are at risk?Nazmus Sakib:Right, right. So much work has gone in to so many different parts of the technology stack, right? You think about the work that we've done at Microsoft and across the industry around things like antivirus solutions. You look at modern platforms like Microsoft ATP, Advanced Threat Protection, where you have just a view of the health of your operating system across many devices that's customized for your enterprise. All of those things, in many ways, have already made it harder and are increasingly making it harder for attackers to do things that they would have maybe gotten away with in the past for attacks in the operating system.Nazmus Sakib:And so naturally, when you make one thing harder, you incentivize attackers to go elsewhere, right? And so what we saw as a trend and one of the places where this was really sort of evident to us in a way that felt it wasn't just us looking at it, it was also externally reported is if you look at the NIST which is the American standards body, essentially, the National Institutes of Standards and Technologies, I think, I'll have to go verify that, but they actually maintain the National Vulnerability Database. So if you think about vulnerabilities that get reported, you see in the news and they often have some numbers associated with it. That's actually all the numbers in the National Vulnerability Database. Nazmus Sakib:And so one of the things that you saw in the research that's being done in the industry, this is where all the security researchers report issues. It's like the aggregate. This is how the industry keeps track of all the vulnerabilities that are happening across all technologies. There was a large spike in firmware. If you just go to the NIST website and you go type into firmware, it went from a handful of firmware vulnerabilities being reported in, I think, 2016/2017 to hundreds being reported in the last year or two. And so a huge spike beyond exponential. And that really is because we're making it harder to do the things that perhaps attackers would be able to do in the past and the operating system. And so people are naturally moving elsewhere. And so they're gravitating towards firmware as an avenue. So that's one reason.Nazmus Sakib:The other reason is coming back to what I was talking about in terms of how a platform boots. Firmware, because it's the first thing that runs on your hardware, because it needs to, just by its very nature, set up your hardware in the right configurations, it actually bootstraps a lot of the security on your system. Right? And so it's almost like a double whammy. Attackers are moving to a place where a lot of the problems that have been solved in the operating system from a security perspective, they're trying to work around those protections. And then in firmware, they actually see that you have this highly privileged environment firmware typically has almost usually when it starts up, almost unrestricted access to all the hardware and the data that's on your hardware. And so that's really where we're seeing this trend where attackers are... the security researchers suggesting that attackers are going to be moving there. Nazmus Sakib:And one very recent practical example of a threat where these trends are bearing out is just, I think, last week, there was a report that TrickBot which is almost like a modular malware that's being used in a lot of other ransomware attacks, it's actually added firmware capabilities. So it's using other longstanding well-known vulnerabilities in the operating system, but because of the trends I've just described, we're seeing TrickBot add new firmware attack capabilities as well.Nic Fillingham:Sakib, do we know when firmware attacks begin? Is there a defining moment in time when firmware became an actual viable target? Or has it sort of always been there and it's just recently evolved?Nazmus Sakib:It's always been there. I mean, firmware is always run with high privileges in a way that it may be difficult for operating system software, including security tools, to tell what's going on in firmware. It's easy for firmware malware to hide what it's doing. But if I were to think of a tipping point, if you will, a couple years ago, we saw that at least one example of what's typically associated with a particular nation state threat actor. There were targeted attacks a couple years ago that were using a firmware vulnerability. So in some ways, that was a very clear signal that not only is the security research headed that way, but there's at least that first example. It's almost like the canary in the coal mine, if you will, where we saw an example of an attack that tried to do exactly what I described, is use for a very targeted attack, use firmware to circumvent a lot of the security tools, and find a way to persist. Nazmus Sakib:And with developments like what I talked about for TrickBot, which is generally often used by many different actors trying to orchestrate different ransomware attacks like Ryuk and Conti, we expect to see that trend sort of increase. And so if I were to think about that first tipping point where attacks start to become real, the LoJax attack is, I think, what it's typically referred to as maybe the one I can think of where it really sort of became not just a trend we're seeing in the research, but a really practical attack. Nazmus Sakib:By its very nature, firmware is complex. There's tens of thousands or millions of lines of code running if you think about all the firmware that runs on your system. So if you just think about the basic security principle of trying to reduce your attack surface, trying to have lease privileges, what you really want to be able to get to is that your trust is not necessarily fully dependent on all the firmware being written totally correctly and totally secure and not vulnerable to an attack. Ideally, you want to not trust that huge infrastructure. You want to be able to go do that trust of fewer set of things. And that's sort of the journey that we've been on recently with our OEM partners as well with secured-core PCs is to do that evolution. A UEFI secure boot doesn't go away. It's still an important technology. But we want to be able to start layering on additional capabilities that can start to protect important security properties or security capabilities even from firmware compromise as that's really where the trends are going from an attacker perspective.Natalia Godyla:So your team has done a lot of great work around secured-core PCs. What would it take for an attacker to actually break into one? Is it possible? What do they have to overcome?Nic Fillingham:Without obviously giving away some operational security here, but just like in Bizarro fictional land with infinite compute power and physical access to the device, what are the monumental challenges that would need to be overcome?Nazmus Sakib:There are a couple places that I think are interesting that we're definitely thinking about. Security is not a static thing. It's always dynamic. We do something and then so do attackers. And so if you think about... It comes back to maybe the foundation analogy. We are building a lot of our security promises on things like the TPM. We want to be able to securely record the firmware that's running so that we can actually tell that it's the firmware that we expected. Right? So that's an area that we're thinking hard about and it's part of the motivation for Pluton. I'll leave it up to you all to interrogate Peter around what the effects are, but I think that's one place where a lot of our security promise is built around that.Nazmus Sakib:We spend a lot of time thinking about TPM attacks. And it's a big part of the motivation for why we're adding another choice to the Windows ecosystem around using Pluton, is just being able to continue to raise that bar against attackers. So I'll leave it to you, Nic and Natalia, to interrogate Peter as to how Pluton will help with the security of future Windows systems.Nic Fillingham:We'll absolutely do that. So Sakib, thank you so much for your time. As always, we will have some notes. We'll have some links in the follow-up show notes. And I'm not sure we've actually offered this to listeners before, but if you do have questions about securing firmware, anything that Sakib talked about, contact us on the Twitters. You can send us an email,, and we'll do our best to point you in the right direction. Thank you much, Sakib.Nazmus Sakib:Yeah, no. Definitely thank you for having me on here. It was just a great conversation. I enjoyed it. And I second what you just said. We'd love to hear from listeners around things that we can do a better job of communicating or feedback folks have on how well we're doing in terms of meeting their needs. Nic Fillingham:Sakib, thanks so much for your time, mate.Natalia Godyla:And now, let's meet an expert from the Microsoft security team to learn more about the diverse backgrounds and experiences of the humans creating AI and tech at Microsoft. Today, we have Bhavna Soman on the episode. Thank you for joining us.Bhavna Soman:Thanks for having me, Natalia and Nic. I'm very excited to be here right now.Natalia Godyla:We're excited to have you. So love for our audience to get to know you a little bit more. What is your role at Microsoft? What does your day-to-day look like?Bhavna Soman:Yeah, absolutely. So my official title is senior security research lead. But like it often happens in big organizations, it kind of doesn't accurately reflect what I do. I lead a team of security researchers and data scientists who use machine learning and AI to fight threats on the Microsoft Defender platform. And that kind of reflects my own background as well, which has been checkered with experience in security research and machine learning. So to me, that's a very good fit even though I can't get them to include all of it in my title.Nic Fillingham:Bhavna, we've spoken to a few of your colleagues on the podcast already: Holly Stewart, Jeff McDonald recently, Karen Lavi. How would you describe what you do? What is different about your role and your team compared to maybe Jeff's team or Karen's team, et cetera, et cetera?Bhavna Soman:Yeah, absolutely. So the focus for my team is on using AI and ML on building intelligence and context for our enterprise customers. So when you look at how you want to apply machine learning in data science, I think it all really boils down to how can you reduce the dependency on human beings who have the security expertise? How can you bring in AI to help enterprise customers better defend themselves in this field that has a scarcity of talent, to be honest? And so what they do is look for clean or malware files. Whereas my team is focused on providing, for example, information about emerging campaigns or information about, what are the attacks that are linked to each other and form one incident so that an organization can address them together as a whole and therefore get efficiencies from that analyst as well?Bhavna Soman:So these are just a couple of examples of what I mean when I say like we provide the intelligence. So I think someone put it very succinctly a few weeks ago where Jeff's team finds the badness, Karen's team finds the goodness, and I kind of bring it all together and give it meaning.Natalia Godyla:That's awesome. I love that definition. Nailed it. And stepping back for a moment, I'd love to hear about what brought you to Microsoft and what brought you to security research. As you mentioned, you had a journey that included machine learning and security research. So how did both of those come into your career path?Bhavna Soman:So I was always excited by security. And even from a very young age when we had our first laptop, which was like way, way back. I think it either had Windows 95 or 98. So it was really old. And those days, you get infected by stuff all the time. So for my family, it used to be my job to kind of figure out exactly where was the registry key in which this thing had saved its autorun tactic or persistence tactic. And at that time, I didn't know what any of these were called or anything. But that's how I first got into it. And then I decided that I really loved this sort of adversarial aspect of security. It really brings an excitement to the whole thing for me. Bhavna Soman:My path did not take me directly to security still. My undergraduate studies were in mechanical engineering. So thankfully, I got a fair bit of math and also programming classes in, but I was chasing different things at that time. But after a while of working in that space, I was actually doing pipeline design for this company that constructs oil refineries, which was a very soul-sucking job for me. Yeah. I didn't like it at all. I did that for two years after college, and it just was not for me. So I was like, "Okay, I really love computers. I have to go in that direction." So I started to build software tools for that company. And then that gave me sort of this way to dip my toes in. And then I realized that, okay, this is definitely something I love doing. So I decided to go for masters. Bhavna Soman:And then when I was choosing my area of focus for my masters, I was like, "Yes, security has to be it." So I went to Georgia Tech to do my masters and I specialized in security. So that gave me a great sort of grounding and all of the basic skills, a great background at the industry. And Atlanta has a very good infosec community too. So I had the chance to get plugged into that. Yeah. I really loved going there. And after my education there, I worked for this startup out of Georgia Tech, which incidentally specialized in using machine learning for network security. So that's where I think I got introduced to, hey, machine learning and artificial intelligence can have something to say about this. Bhavna Soman:The more I stayed in the security industry, this problem of how it's all a whack-a-mole where a few people are chasing thousands and millions of different variants of the same attack. It really impressed on me that this is not something I can do manually. I can reverse 10, 15 samples. I can't do a thousand. So that's where the power of AI and machine learning really struck me. So I think that's where I started going deeper and deeper into that.Nic Fillingham:I wanted to come back to something that you touched on about being the family... What did you say? When a virus came on the computer, you would be the one that would be in charge of getting it off? Is that correct?Bhavna Soman:Yeah. Yeah. So at that time, I think they weren't super severe viruses. They weren't doing human operated ransomware stuff. For instance, they'd show you annoying pop-ups or they would change your search engine all the time. And they were doing very annoying things like that. I took on the task of investigating, how exactly is this thing coming back even though I deleted it? And then I started to discover the hidden mode in Windows and I started to discover all of these registry keys and rededit. It kind of went deeper and deeper and deeper from there.Nic Fillingham:Got it. Were these in the days where you could just install as many toolbars as you wanted inside your browser to the point where you could no longer see a web page? Are we going back that far?Bhavna Soman:Yeah, yeah. It was one of those days where... And also, Google was not really a thing. I remember Yahoo chat rooms used to be the big thing.Nic Fillingham:AltaVista, baby. AltaVista.Bhavna Soman:So fun times. There was a simpler world for sure.Nic Fillingham:Bhavna, how long have you been at Microsoft now?Bhavna Soman:It's been three and a half years now.Nic Fillingham:Got it. And and the first role that you came into at Microsoft, was that in the team that you're in or was that in a different group?Bhavna Soman:It was still with Microsoft Defender, but I was doing slightly different stuff. I was focused more on just pure security research and not as much on the machine learning and AI aspect.Nic Fillingham:Three and a half years ago, what were you focused on? And how has that sort of potentially evolved? How has that changed today? Were you still focused on the same types of attacks? They've just sort of evolved in sophistication. Or was it a completely different world three and a half years ago?Bhavna Soman:So when I first came to Microsoft, I was coming fresh off of Intel. At Intel, my focus had been on threat intelligence. Again, this was back when threat intelligence was just starting to become a thing. So I joined Intel before that. And at that time, they needed a threat intelligence platform where you can gather all of the TI information from all these feeds: internal, external, et cetera. So I built that first platform, plugging it into all the internal/external data feeds, organizing the data, and then having that pumped into the various prevention and detection systems. So that's what I was doing primarily at Intel. So when I came here at first, I was still in that mindset, and I was still trying to apply intelligence to improve protection. So I was doing a lot of hunting on VirusTotal, kind of try to find out where our biggest gaps were, and trying to plug those. Bhavna Soman:But very quickly, that pivoted to using machine learning for security was focused on non-PE files. So very heavily focused on the document files that we very often see come in as email attachments, and then they will lead the user to download something actually bad like, again, an Emotet or Dridex or something. So it was very focused on those macro files and other non-PE files. JavaScript was a big one at that time. So writing classifiers to differentiate between malicious JavaScript and the benign kind. Those were some of my first projects here.Natalia Godyla:So you said a couple of times that the draw of machine learning for you is the potential for scale, the potential for helping to fill that skills gap. So as you're shifting into roles where machine learning is playing a bigger and bigger part, what are the achievements that you're focused on? What would you like to try to automate better so that humans can shift to other tasks?Bhavna Soman:So there is one problem, which is very close to my heart. And that is the problem of the core threat intelligence business. So Microsoft Defender has a really big threat intelligence team. And this was something... I was part of the threat intelligence team at Intel as well. And all through my time working with these teams, it's been obvious that threat intelligence is very manually driven right now, right? It has to be a human that is reading files or PDFs or white papers. And then this human is, again, observing traffic data whether by hunting or through the attacks that they are remediating or something like that. So this human is then kind of assimilating all of these insights that they have about these attackers. And then they put it out somewhere. Like maybe they will communicate it to their customers saying, "Hey, this is what you need to be careful about." They may write a white paper or they may do detections as a result of that. So this is a very human thing. Bhavna Soman:And when I look at artificial intelligence and machine learning, to me, using large amounts of data to extract a few critical insights, to me, this is a very good use case for machine learning and AI. So this is a problem that I have been working on for a really long time. My first attempt at this was while I was at Intel, and I did this kind of cross-team project with a team that was in Argentina at that time to work on a method that could use question answering techniques from machine learning to answer questions about attackers. So if I had a question about, "Okay, what is the tool that this attacker uses? Or what is the victim vertical for this attacker?" Can I use question answering techniques and train on the corpus of data available about these attackers and have an AI-based system give an answer? Bhavna Soman:So I've been attacking this problem for many years. My first attempt while I was at Intel was not very successful. But a couple of years ago, I gave it another shot. And this research ended up being... I presented this at Black Hat last year where I was talking about how we can use some new techniques that had come out since then around word embeddings, natural language processing, and domain specific named entity extraction to do similar stuff. So I think I've been making progress on that problem. And now I'm working on a project with University of California, Berkeley on this security AI RFP where now they're expanding some of this work into the security knowledge graph where their aspiration is even bigger. Yes, we grab all of this data from a variety of different data sources. Yes, we do named entity extraction. But what else can we do on top of that? Can we automatically build, for example, YARA signatures based on this? Can we use multiple data sources to achieve consistency internally within this graph? Bhavna Soman:So that's where we're seeing AI and machine learning will take threat intelligence and help it become a little bit less manual, and again, less dependent on manual expertise?Natalia Godyla:What challenges are you facing with achieving some of the goals you've outlined? I'm assuming compute is always something that's in the back of your mind. What else would be a barrier to potentially achieving some of these successes? Or what are you tackling right now to reach your goals?Bhavna Soman:That's a great question. Compute is a big one because on one hand, we have large amounts of data. But on the other hand, A, to process all of that in a deep learning style would take huge amounts of compute that would make our product run very inefficiently on our clients and in organizations' machines. So usually, that's not feasible, which is why one of our big focuses is to find efficiency in whatever techniques we're using so that the model can be lightweight and yet perform with similar degrees of precision and recall. Bhavna Soman:Another big challenge we face is good labels or ground truth. Just because the spectrum of badness is so huge, on one end, you have these just adware things are grayware things that their whole goal might be to show advertisements or cause pop ups. And on the other end, you have APT threats. So in this wide spectrum, we have to find good labels for a large enough set for each particular category so that we can accurately classify threats and inform users about that. That's been a very interesting problem too. Going back to the threat intelligence space, one really huge challenge is that the field is continuously evolving. A particular thing might be used for human operated ransomware on day one, but on day 30, it's hosting some random adware or some software bundle or something. So within that span, even in shorter spans, the situation really changes. The intelligence you have really changes. So all of your machine learning systems have to be able to constantly getting the latest information adapting to that. So those are some of the big challenges we face in this field that we're trying to work around.Nic Fillingham:Bhavna, one of the questions we like to ask on the podcast is, what from your personal life, whether it's a hobby, whether it's something growing up as a kid, whether it's education or previous job, do you bring forward into your current job that could be considered maybe unorthodox? You teased very early on that maybe you play D&D. Is that true? Bhavna Soman:Yeah. I play video games or board games. I'm into all of that.Nic Fillingham:Is that a passion for you? Do you find yourself bringing any game theory or the way that you would approach a D&D encounter into your day job?Bhavna Soman:I think my biggest influence is books and language. I have been into books as far as I can remember. That was my favorite birthday gift when I was a kid. I just dragged my parents to the bookshop and buy a bunch of stuff. And a peculiar way in which humans use language and give meaning to it, to me, that is a source of endless fascination. Which is why one of the favorite authors for me is Patrick Rothfuss and his book, Name of the Wind. I think that book really talks about... It's a fantasy book. So it kind of goes into like if you know the name of a thing, then you have some control over it. It's a philosophical point, but also it says something about language. And in my mind somehow, all of that comes together and that really leads me into, how do machines interpret language? What does it mean for a machine to understand language? And when we're building all these natural language processing models, what exactly are we doing? And then what exactly are we missing from what human communication actually entails?Bhavna Soman:Which is why I'm kind of always drawn into this threat intelligence field because I'm like, "This is really where the importance of language and communication becomes connected to security." So that's kind of this one thing for me that I really, really love. In fact, one of the really cute examples that's always stuck with me is when you do a beginner course on natural language processing, you always kind of get this example. It's called crash blossoms. There was apparently a headline in the newspaper a long time ago where the headline said, "Violinist in Japan Airlines Crash Blossoms." And obviously, the headline meant to say that this violinist who was involved in this air crash a while back is now doing well. But when an NLP based system is trying to process it, it is like, "What is crash blossoms?" And I love that problem because it kind of emphasizes very clearly how machines are different from human beings, and yet how we're trying to bring the two closer for our own benefit.Natalia Godyla:I feel like one of the other unique points about language is just the evolution of slang. So I'll be curious to see how NLP processes and consumes slang because that is such a cultural moment. It depends on the cohorts of people that you surround yourself with, the social context. Bhavna Soman:Yeah, that's a great point. You talked about slang specifically where a meaning of a particular word or phrase can be different based on even the environment or the forum in which it is used. Certain terms, if you use it in an industry specific way, will mean very different than in the general sense. And we come across that in security so much, right? We have all these actor names like Scary Panda or Crawling Spider. And if you think of using like a traditional NLP model and all of this data, you're like, "This is not going to make sense because you're talking about a specific entity, an actor, not an animal." So we do have those kind of challenges in our domain. And I love diving deep into that.Nic Fillingham:So I have another sort of random question. I was possibly laying the ground for this with my previous question about, what from your hobbies do you sort of bring forward into your work? Your avatar, your photo in the Microsoft GAL in our sort of identity system is Megamind. Is that right?Bhavna Soman:That is absolutely right. I think that really ties into my sort of chaotic neutral rogue character because Megamind is a really good example of that, right? Supposed to be a villain but is a hero, but also is a villain in some ways still. This was actually a prank. We had Microsoft Month of Give last month. So your teammates could donate some money and force you to change your profile picture. So that's what I got.Nic Fillingham:Did you choose Megamind or Megamind was thrust upon you?Bhavna Soman:I chose Megamind. I was like, "Okay, this is the most appropriate for me."Nic Fillingham:Oh, so you do resonate with the Megamind character on some level?Bhavna Soman:I do. Yeah. I think so. And also, it's a really good movie that kind of has not had its time in the limelight for a while.Nic Fillingham:I don't know if I've seen it. I think my kids have seen it. That's sort of why I know it because I think I've sort of had to approve them watching the movie, but I don't think I've seen it. It's good, is it?Bhavna Soman:It is amazing. You should definitely watch it. It's a very cute movie.Natalia Godyla:I think we have our homework, Nic. I haven't seen it either.Nic Fillingham:Bhavna, before we let you go, is there anything you would like to plug? Any sort of organizations you're a part of? Any communities, groups? Anything you'd like to say out there to aspiring students of machine learning who either want to get into the field or just want to get better at machine learning?Bhavna Soman:I would love to. So the organization that I want to talk about is not associated with machine learning only. It's associated with security all up. So I am part of a group of women called BlackHoodies. And we are committed to increasing the participation of women in hard technical areas, which sometimes don't see as much participation from minorities. We are across the globe across many companies group. The only I think criteria is you are a woman, whatever your definition of that is, and it's always free. We hold classes at multiple conferences across the world which we'll do things like reverse engineering, Windows, ARM, web hacking tools like Ghidra, all of that. We have all these trainings that are completely free. And now that we are in the pandemic, we're doing some of these remotely. So please follow us on Twitter. And if you're interested in joining one of these trainings, it's super easy. And we really, really welcome anyone who wants to learn about this stuff.Nic Fillingham:As you were talking, I searched Black Hoodie on Bing and just got a thousand results for buying a black hoodie. What is the URL for the community group? I think I may have just accidentally purchased a black hoodie. I've got Amazon, what is it, one click buy. I went a little too quick. I was trying to pay attention to the recording window for the podcast and then searching for what this was. Anyway.Bhavna Soman:I hope it fits. So the website is And we talk about all of the latest events or workshops that are happening there. Usually, when Microsoft holds Blue Hat, we'll do a bunch of trainings at Blue Hat as well. I do the beginners reverse engineering for x86 as part of that. But right now, we don't have in-person conferences, but we're doing virtual stuff.Natalia Godyla:That's great, Bhavna. I think one of our previous guests has also shared BlackHoodies. So thank you for highlighting it. It sounds like a great organization. And to our audience, please check it out. Thank you, Bhavna, for being on the show with us today.Bhavna Soman:Thanks for having me. It was super fun.Natalia Godyla:Well, we had a great time unlocking insights into security from research to artificial intelligence. Keep an eye out for our next episode.Nic Fillingham:And don't forget to tweet us @msftsecurity or email us at with topics you'd like to hear on a future episode. Until then, stay safe,Natalia Godyla:Stay secure. See for privacy and opt-out information.
Watchdogs in tow, hosts Nic Fillingham and Natalia Godyla are joined by guest Randy Treit, Principal Security Leader at Microsoft, to examine the process of identifying the source of a threat and stopping the spread by protecting “patient zero.” Randy has a few key tricks up his sleeve as a defender, but you can decide if they’re more impressive than the antics he and his identical twin have pulled while working at Microsoft.In the second segment, Jeremy Dallman, Principal Program Manager at Microsoft, discusses why some bad actors are known in the security world under some of the most seemingly harmless codenames, such as “Fancy Bear” and “Charming Kitten”, and highlights the techniques his team is using to protect Microsoft’s customers from Nation-State actors.  In This Episode, You Will Learn:  • How Microsoft is defending and protecting patient zero• The history of Defender and antimalware • The process of finding gaps in protections • The importance of protecting customers from Nation-State actors • How and why security vendors use codenames to refer to threat activity groups  Some Questions We Ask:• What is different about focusing on patient zero than other aspects of security?• How does Microsoft measure the false positive rate in protecting patient zero?• What tools are being used on a day-to-day basis in defender security?• Why does Microsoft partner with the industry to identify Nation-State actors?• How many groups are utilizing AI and ML to enhance their ability to become a threat?  ResourcesMicrosoft Digital Defense Report: Randy’s LinkedIn Jeremy’s LinkedIn Nic’s LinkedIn Natalia’s LinkedIn Microsoft Security Blog: Transcript(Full transcript can be found at Nic Fillingham: Hello, and welcome to Security Unlocked, a new podcast from Microsoft, where we unlock insights from the latest in news and research from across Microsoft security engineering and operations teams. I'm Nic Fillingham. Natalia Godyla: And I'm Natalia Godyla. In each episode, we'll discuss the latest stories from Microsoft security, deep dive into the newest threat intel, research and data science. Nic Fillingham: And profile some of the fascinating people working on artificial intelligence in Microsoft security. If you enjoy the podcast, have a request for a topic you'd like covered or have some feedback on how we can make the podcast better. Natalia Godyla: Please contact us at or via Microsoft Security on Twitter. We'd love to hear from you. Hey Nic, how's it going? Nic Fillingham: Hello, Natalia. It's going well, thank you. Welcome to episode 10 double digits. It feels like a milestone. That's a milestone, right? Natalia Godyla: Heck, yes. I think we were proud of ourselves after episode two. So I feel like this feels a little bit more legitimate, a good start to 2021. Nic Fillingham: Great start to 2021. But we were talking, just before we started recording and there is some sad news. Natalia Godyla: Okay. So to listeners that had heard and loved our story about the Somerville Turkey, of course. The Somerville Turkey is no longer, so the Somerville residents had fed the turkey and the turkey became aggressive as a result. And it is no longer a hallmark of our city. Nic Fillingham: The problem was they fed the turkey pure creatin, that was the issue and Red Bull. Natalia Godyla: They didn't publish that in the news story, they're trying to keep that hash, hash. Nic Fillingham: That's why it got aggressive. But no, if you have no idea what we're talking about on our Thanksgiving episode, Natalia told us about a famous turkey in Boston that has a name and it's got an Instagram page or something like that, but unfortunately it's no more, it's pretty sad. Natalia Godyla: Now that the turkey is no longer, maybe we should memorialize it. Nic Fillingham: Ooh, so you're thinking we could potentially adopt the Somerville Turkey as our Security Unlocked mascot. Maybe we could create some kind of small statues, some kind of plush toy, is that where you're going? Natalia Godyla: For some reason, my immediate image was a butter sculpting contest, in which we sculpted butter sculptures of the turkey. Nic Fillingham: Hang on, what? So, I had said as a mascot and something, I think I said the word swag, at least it was in my brain. So something we could send to listeners, and so I just immediately jumped to the logistics of how do you send butter through the US Postal Service in an intricate shape, like that of a turkey? Natalia Godyla: Yeah. I don't think you should be taking my suggestions quite so realistically, I mean- Nic Fillingham: If we had to choose though, between memorializing the Somerville Turkey and our previous plan which was the mighty alpaca as our animal mascot, where are you leaning? Natalia Godyla: Alpaca. Nic Fillingham: Can we justify that from a security perspective? Is there any security link whatsoever from either a turkey, Somerville Turkey or an alpaca? What are you looking up? You're looking up something right now. Natalia Godyla: I'm looking up facts about alpacas because I have to be honest, this is purely on level of cuteness for me. Nic Fillingham: Okay. So our Executive Producer, Bruce Bracken has just chimed in saying that god llamas and god alpacas are a thing. So it says here that a god llama, alpaca or hybrid can be used in farming to protect sheep, goats, hens, or other livestock from coyotes, dogs, foxes, and other predators. Ladies and gentlemen, we have a winner. We now have a solid link from the alpaca to security. Well done everybody, congratulations, mission accomplished, we can go home now. All right, beautiful. Natalia Godyla: On a minimum, we can talk about our next episode. Nic Fillingham: Absolutely. All right, so let's table that. We've decided it is going to be the alpaca because the alpaca can be employed as a rudimentary guardian of livestock. But speaking of the podcast, on today's episode, first up we have Jeremy Dallman joining us from the MSTIC Group. I'm not going to explain what MSTIC stands for because Jeremy will talk about it. And it's a great start to the conversation. Jeremy is coming on to talk to us about the nation-state section or chapter in the Microsoft Digital Defense Report, the MDDR, this is the third of five conversations that we're going to be having on Security Unlocked, where we deep dive into some of the topics covered in that report. Nic Fillingham: This is also I think, the first time that the MSTIC team have compiled a lot of their nation-state tracking activity over a sort of 12 month period into a single report. So first of all, it's a great read, make sure you download the report, And then, it's a great conversation with Jeremy who really helps us sort of understand some of the core principles and ideas around sort of why is Microsoft in this space, and then sort of what does Microsoft do with tracking nation-state actors. And then after Jeremy, we talk to- Natalia Godyla: Randy Treit, a Principal Security Researcher at Microsoft, a long time employee at Microsoft who has seen a lot of different groups and brings that expertise to his security team today. So we're talking to him about his path to security and he is another security professional who doesn't have a formal or standard path to security. So he doesn't have a formal education. And I think it's a good testament to the fact that so many security folks are autodidactic and just have a love of technology and find themselves continuously passionate and interested in it, and eventually get to do their passion for a job. Nic Fillingham: On with the pod? Natalia Godyla: On with the pod. Nic Fillingham: Jeremy Dallman, welcome to the Security Unlocked podcast. How are you doing?Jeremy Dallman: I'm doing great guys. Thanks for having me. Nic Fillingham: Thank you so much for coming on the podcast. This is one of several conversations we're going to have with folks that have contributed to the Microsoft Digital Defense Report that was released in September of 2020. Jeremy, thanks for coming on. You're going to talk to us about chapter two, which is the chapter that talks about nation-state threats. This is going to be a fascinating conversation. I'm really, really interested and excited to hear what you've got to tell us. But can we just start a little bit with, who are you? What's your job? What do you do at Microsoft? What does your day-to-day look like?Jeremy Dallman: Sure. So let's see, in Microsoft terms, I'm a Principal Program Manager, in the Microsoft Threat Intelligence Center. We call ourselves MSTIC. So I'll probably use that term off and on throughout the conversation, it's much easier to say it than Microsoft Threat Intelligence Center. As a Program Manager in MSTIC, I am responsible for, let's see, directing a large number of projects that kind of span incubation and driving threat intelligence initiatives, both in MSTIC and across Microsoft.Jeremy Dallman: I do things around building and creating strong collaboration partnerships across the security industry, because malicious actors, like nation-state actors, don't just target Microsoft. I also work on sourcing the best possible tooling for our analyst and managing all of our public facing messaging around MSTIC and the threats that we track. So I guess in general, my role is always looking for ways to improve how MSTIC protects our customers, making sure that the analysts are successful and effective at hunting. And making sure that MSTIC knowledge outside the company is communicated effectively to protect our customers and enable better protections across the ecosystem. Nic Fillingham:I have ask, is MSTIC a backronym? Did you guys get in a room and say, "How can we come up with the coolest acronym in the company, and then make it work for what we do?"Jeremy Dallman: There's actually a couple of others I think, that are cooler, as well though. Nonetheless, no, our GM is notorious for let's just say, obscure acronyms that translate into words. So it took a little bit of effort, it took a little bit of time, but we came up with Microsoft Threat Intelligence Center and M-S-T-I-C pronounced as MSTIC. So we worked through a few other variations, but I think this was the best one that came out and it seems to have stuck. Nic Fillingham: I think there needs to be an offshoot team for analytics and learning at the end. Does anyone get that-Jeremy Dallman: Yes, Nic. Yes, yeah. Nic Fillingham: Okay, good.Jeremy Dallman: I know a couple of people on the analytical side that might actually run with that, I might have to jot a note down. Nic Fillingham: There you go, you can have that one for free, no royalties from me, that's fine. Natalia Godyla: The next one's charged, though. Nic Fillingham: The next one's not free, this first one's free. So Jeremy, you're going to walk us through chapter two, the nation-state threats, it's a pretty lengthy section of the MDDR. It's also, I think, correct me here, this is the first time that we've done sort of an annual wrap-up of what Microsoft has seen on the nation-state space. I think obviously, we've had lots of blog posts and activity over the many years on the activity, that we've seen and sort of how we've contributed to it. But previous sort of security intelligence reports didn't really include a lot of nation-state activity. I mean, correct me if I'm wrong here, but is this sort of the first time that we've done an annual look back at what happened in the nation-state space?Jeremy Dallman: Historically, our team hasn't been very publicly outspoken and we haven't really, historically didn't spend a lot of time talking about what we've done externally. So this is definitely unprecedented and something that's brand new for our team. It's kind of along the lines of what we've been doing over the last couple of years, talking a little bit more publicly about threat actors and such. So I think this is a fantastic roll up in view of what we do. I think it goes along with our expansion of MSTIC as an organization and kind of what we've been trying to do, informing our products and customers more broadly. Natalia Godyla: So Jeremy, why does Microsoft do this work? Why do we partner with the industry to identify nation-state actors?Jeremy Dallman: Sure. I think the short version is that Microsoft customers using our products are often the target of nation-state actors. And those customers expect Microsoft security products and Microsoft to help protect them from those threats. So MSTIC tracks nation-state activities to protect our platforms, to protect our services and protect our customers from those more sophisticated threats. Nic Fillingham: So, Jeremy, I've got the report open here in front of me and for those playing along at home, you can download the report. It's the Microsoft Digital Defense Report @ And if you scroll down to page 44, there is a really interesting sort of graphic here. It says, "The sample of nation-state actors and their activities." And there's a bunch of what look like sort of chemistry symbols from sort of the periodic table of elements with a lot of chemistry names and symbols. And then there's some sort of other things as well. Can you sort of walk us through, what are we looking at here? Is this actual sort of nation-state actors and sort of how they're referred to? And the names that are being used to refer to them?Jeremy Dallman: Across the security industry, a number of different security vendors use different code names to refer to sets of activity that are tied to certain actors or sets of activity groups. So we use code names because we can't always necessarily tie that to a specific country, or we may want to do attribution. Other security vendors will use kittens and tigers and bears, some use numbers and a variety of different code names. And at Microsoft and in MSTIC, when we were trying to figure out how we were going to do code names, we tried a bunch of different things. I think initially, there was some use of dinosaur names, that got fairly complicated and hard to pronounce fairly quickly. I think we played around with a bunch of other things. At one point, I recall we were looking at flavors on the beer flavor wheel, I'm not sure there was enough of them.Jeremy Dallman:So we played around with this a little bit and we ended up basically at periodic table of elements because there's not really a licensing violation there, so we didn't need to worry about that. And there was plenty of them and they were fairly unique. So we code name our actors by elements in the periodic table. And we will name an actor, an element, once we understand that actor has a unique set of activity. But on that page 44 in the report is a summary of a few of our key activity groups via their element names. And largely focusing on the four regional sets of actors that we, and most threat intelligence teams will focus on, Iran, China, North Korea, Russia. Nic Fillingham: And is there any sort of logic to the particular element that's chosen? I mean, I noticed that there's no hydrogen, there's no oxygen. Well, they seem to be up towards the top end of the periodic table. I've never even heard of-Jeremy Dallman: Yttrium? Nic Fillingham: Yttrium? Did Kanye West come up with that one? What's that?Jeremy Dallman: No, it's kind of funny because we actually have an individual on our team over in our UK office. She's responsible, she's our librarian, is kind of the role that she plays and she is responsible for naming. So I don't think there's any specific logic or pattern to who gets what name. I don't even know if our analysts have a say in picking any of the names, but our librarian is the person who basically gives these names out. And I don't think she has any set structure or method for picking the names. Nic Fillingham: I was really hoping you were going to say there was a periodic table of elements stapled to the wall, and then you had to start with dots. Natalia Godyla: Somehow, I knew I was going to be dots.Jeremy Dallman: You know what? I honestly would not be surprised if that was actually the case, but I can't verify that. Nic Fillingham: All right. Well, that's for another episode of the podcast for us to follow up on. Natalia Godyla: So can you provide a little bit more context on the players? What do we know about them? Their motivations? Infrastructure?Jeremy Dallman: Sure. So a number of these actors are pretty well known. When you talk about kind of the more popular, more widely discussed actors, it's kind of hard to not fairly rapidly, get to Strontium, which others refer to as APT28 or Fancy Bear. And this is an actor set that we believe originates in Russia. Jeremy Dallman: This is someone that we've... an activity set that we've talked about fairly extensively over the years of public discussion around these actors. Whether targeting individuals or campaigns or entities involved with politics. So they're probably the more well known out of Russia. I'll just kind of hit a couple in each one of these here.Jeremy Dallman: Phosphorus, which is an actor set that we believe is originating from Iran, also known as APT 35 and Charming Kitten. They're well known for targeting government defense industrial, especially in the region, in the Middle Eastern region. Especially fond of targeting personal email accounts and going after personal email accounts as a way to gain access to systems that they're targeting or individuals and surveil individuals. A lot of activity there tied to sanctions and research around policy, that sort of thing.Jeremy Dallman: In China, we have actors that more broadly, I would say are more likely to use more sophisticated technical solutions. Trying to bypass or using more sophisticated malware, but technology, supply chain targeting, targeting education and medical research. Actors like Barium known as APT 41. Manganese, which will often target communication infrastructure. They'll even go after things like satellite or defense industry or GPS navigation.Jeremy Dallman: And then North Korea actors like Thallium and Zinc. We'll see them targeting human rights organizations and surveilling human rights organizations that might be involved in their region geographically. But we'll also see them often targeting think tanks and governments that are involved in sanctions or policy decision-making that might be tied to the Korean peninsula. Nic Fillingham: Why is Strontium a nation state actor and not simply just a sort of independent group of baddies?Jeremy Dallman: No, that's a great question. I think the simple definition of a nation state activity group is we defined it as cyber threat activity that originates in a particular country with an intent to further national interests. So because that activity fits that parameter, there's an assumption that it's more well-funded, potentially more sophisticated. And they'll more likely going to be using what we call advanced persistent threats which is an adversary that possesses a sophisticated level of expertise and significant resources that allow it to achieve its objectives using a lot of different attack vectors. It's a combination of expertise and significant resources, adequate funding to achieve specific objectives in a particular country with intent to further the national interests. Natalia Godyla: And what about attack techniques? So you hinted at that in your definition. So what are some commonalities or patterns that you can identify across nation state actors that differentiate them from other threat actors?Jeremy Dallman: So when you think about nation state actors, and I would say in most of our threats even outside of nation state actors, you're going to see most threats start with email. I think there was a blog post we put out not too long ago that said 95% of threats start with email. Start with an email lure. From a nation state actor perspective, that's largely a technique to achieve reconnaissance. To find out or identify who the people are that they need to target to achieve the objective that they're trying to achieve. So they will do things like password spray techniques to attempt to guess log in passwords for a number of accounts tied to a specific organization that they're trying to target. They will do brute force login attempts, trying to guess the passwords and try to brute force their way into an organization. That early reconnaissance technique allows them to establish an initial foothold into an organization and also then harvest credentials.Jeremy Dallman: So if they can start guessing passwords and they can understand what those passwords might be, they can harvest those credentials, store those credentials and then use those in future operations to come back into that network and execute whatever operation or mission they might be trying to achieve. Once they've actually established in there, and often as a way to get a foothold into a network, they'll use malware. Malware is a very common method by nation state actors. And I would say some actors on the nation state side, because of the excessive funding that they have at their disposal, they will go above and beyond in building up particularly sophisticated malware techniques to bypass common detections by security vendors and some networks. So that's constantly a game that we're playing to understand these malware techniques. We'll also see nation state actor using very sophisticated and personalized lures.Jeremy Dallman: They will spend a significant amount of time. And this is something we just blogged about a couple of weeks ago, an actor named Phosphorus, which originates in Iran. We're actually using building rapport and building relationships with individuals that are tied to international policy. And by building that rapport with those people, they were actually able to send them invitations masquerading as the Munich security conference, which is a prominent international policy conference. Masquerading as the conference and trying to lure that person to their fake invitation so that they could steal their credentials. A little bit of social engineering happening there. But a nation state actor is going to have the resources and funding at their disposal to be able to build out those more sophisticated techniques. And then finally, I would say there's a lot of nation state actors that spend a significant amount of time building out capabilities, relying on common weaknesses.Jeremy Dallman: So when a new patch goes out, patching a security flaw within a Microsoft product, for example. A lot of actors will reverse engineer that flaw. Better understand it then use it to weaponize a new exploit. Which is why it's exceptionally important for customers to patch as quickly as they can to avoid that weakness that Microsoft is attempting to patch. That weakness becoming an entry point for a malicious actor because nation state actors will move rapidly to take advantage of that and then attempt to exploit those weaknesses where they can. So that's a couple of techniques that I would say, like I said, we dive a little bit more into in the report. But there's more in there, especially things like web shell based attacks, which we see increasing, but I'll let you go read that into the report. Natalia Godyla: Yes. Nice teaser for our audience. One interesting point made in the nation state section of the MDDR was the downstream effect. So if I understand it correctly, the nation states will pursue these techniques and then eventually other actors will pick them up. So how does that happen if they are these sophisticated groups that are leveraging, like you said, more complicated malware? Is it that the other attackers use simplified versions of it, or as it's in the wild they get more exposure and are educated on that strain of malware and then are able to use it? So what does the process look like from nation state actor using these attack types to another attacker in the wild?Jeremy Dallman: Yeah, I think you nailed it there with the second example you gave. Because that's typically what happens is once this exploit gets out in the wild it's not just Microsoft watching for these more sophisticated threats. All of the other actors out there, whether they're criminal organizations or individual hackers, whoever it might be. There's a whole ecosystem of people out there that are watching for these threats to evolve and looking for new techniques. So when a nation state actor might have a particularly sophisticated attack that goes out, there's any number of people who will pick up and discover that through various security researchers in the ecosystem. And then they will immediately go do exactly what we do, which is reverse engineer that, understand how it works. And then you'll see variants come out. You look at things like the VPN exploits that came out in mid 2019.Jeremy Dallman: Those VPN exploits were picked up and used by an actor that we call Manganese to steal credentials and gain access to victim networks, using VPN infrastructure and holes in unpatched systems on VPN networks. So when you think about a world, the world we live in right now, where everybody's working remote. And global enterprise IT departments are relying on VPNs to improve connectivity and security for their systems. If that VPN infrastructure is not updated in its patching, actors like Manganese were taking advantage of that patch, reverse engineering it, and then going out to find VPN infrastructure that hadn't been patched and then exploiting it to gain access to those networks. Well, what we've seen subsequently is everybody else saw the technique and realized, hey, VPN, everybody's using those right now. And they started taking that and tweaking the same technique. And now those exploits have become, unfortunately become fairly commonplace. Nic Fillingham: Jeremy, you said that one of the characteristics of a nation state group is the sophistication in their techniques. And so I sort of have to ask, do we know if many of these groups, any of these groups are utilizing AI machine learning? If so, how?Jeremy Dallman: We don't have conclusive evidence I don't think. I mean, short of us walking into their infrastructure and taking pictures of systems, which isn't something we do. But I think there's enough- Nic Fillingham: Why not?Jeremy Dallman: ... indicators. Nic Fillingham: That sounds like a great idea. I'd make that a priority.Jeremy Dallman: That would definitely make our jobs a lot more interesting. I would say that we've seen indication of nation state actors starting to take advantage of whether it's machine learning or AI. It's unclear. They're starting to take advantage of more sophisticated techniques in those directions. When you think about a password spray campaign, where you are trying to attempt to guess the passwords for a number of different accounts across one organization, that takes a certain amount of compute, a certain amount of effort and a certain amount of automation that can be enabled. But if you take that and you expand it into something like we blogged about from Strontium in September, for example. We saw Strontium attempting to password spray a number of organizations, and they were spearfishing hundreds of organizations with thousands of password guesses in very short periods of time.Jeremy Dallman: And then on top of that they were using thousands of IP addresses and anonymization platforms to obfuscate their activity. So when you think about the complexity of that operation and the speed at which they were able to execute it, it would make sense that actors like that are starting to take advantage of machine learning or some automation capabilities on the backend to increase the speed, the effectiveness and the scope of their operations. Natalia Godyla: I think all of this is leading up to what is Microsoft doing? So how are we disrupting nation state threats today?Jeremy Dallman: So we do a number of different things. I would say probably the best and most effective way is using Microsoft's voice to raise awareness of these activities. And that comes in a number of different ways. We have the blog posts that we put out. The Microsoft On The Issues blog puts on a lot of interesting content that's derived from MSTIC research. And what that does is it kind of helps drive that broad discussion around what can be done to combat malicious nation state activity against governments, academia, social organizations, individuals. A lot of nation states like to target your personal email accounts, but we still defend those private email accounts because whether it's Outlook or a personal email account, that's something that we also have to protect our customers who might be getting attacked through that type of a vector. I would say probably one of the more interesting ways has been on the legal side.Jeremy Dallman: So one of our unique ways to target nation state actors has been partnering with our colleagues in the Digital Crimes Unit here at Microsoft. And the Digital Crimes Unit is responsible for pulling together a lot of the evidentiary information and understanding the threats for a legal perspective. And then they take that to courts and use litigation to seize domains and other assets that are being used by these nation state actors. And then actually through legal action shutting down those attack vectors. And then from time to time, we'll also, if we have sufficient information to warrant one time action to delete or shut down infrastructure or assets that are associated with the nation state actor. We'll also take those proactive measures against that infrastructure to basically eliminate visibility or capability on an actor and forcing them to go rebuild that infrastructure. They will typically rollover infrastructure and start rebuilding and come back later.Jeremy Dallman: So that's not necessarily a whack-a-mole game we want to get into in a lot of cases, but if it's for the protection of our customers, or if we feel it's particularly effective, that is something that we'll do as well. So that's a variety of a few ways. Obviously the one that I didn't touch on is probably the most obvious one, is leveraging our own technology and using all the knowledge that mystic collects about these threats, these actors, their tactics, their techniques and translating those into detections. Transforming and putting those into blocks and protections that show up in our security products and protect our customers in their environments. And the whole objective there has always been to make sure that we're implementing relevant, accurate and actionable threat intelligence for our customers. Nic Fillingham: Where can folks go apart from reading the MDDR? Where can they go for more information on how to protect themselves against a nation state attacks if they find themselves in one of these targeted industries?Jeremy Dallman: So we don't have a MSTIC page. I would say in the MDDR, Jeremy Dallman:We definitely have a section at the end of the Nation-States Reference called comprehensive protections required and it walks through to defensive positions that you can take, the strategies that you can enable there. And then at the end of the digital defense report, we have what are called actionable learnings. And I would recommend you go there and dive into that section as well. And every time MSTIC puts out a blog post, we will always have something at the bottom that are generalized recommendations also. So if we put out a technical blog posts that walks through the techniques of gadolinium or strontium, we will always have at the bottom the specific techniques for that threat that would help you mitigate or protect yourself from that threat. So always watch for those blog posts and then probably for the digital defense report. Go out and look at the actionable learnings. That's probably the best place to start. Nic Fillingham: Hey, Jeremy. Thank you so much for your time. This has been a fascinating conversation. We've really only scratched the surface of that nation-state threat section of the MDDR report. So if you enjoyed this conversation, would like to learn more head to and download the report, and there's lots more detail and lots more articles linked too, that you can read to learn more about this space. Jeremy Dallman, thank you so much.Jeremy Dallman: This was fun. Thanks for having me guys. Natalia Godyla: And now let's meet an expert in the Microsoft security team to learn more about the diverse backgrounds and experiences of the humans creating AI and tech at Microsoft. Today we are joined by Randy Treit. Thank you, Randy, for being here.Randy Treit:I'm happy to be here. Thanks for having me. Natalia Godyla: Great. Well, let's kick things off by chatting a little bit about what you do. So what's your role at Microsoft? What does your day to day look like?Randy Treit: My title is principal security researcher. I'm on the Defender endpoint team. So focused mainly on detecting new threats that we haven't seen before. Protecting patient zero is a big focus of mine. Recently I've started looking into some new kinds of attacks using OAuth phishing. So that's sort of my current main focus area, but I've done a lot in the cloud protection. I've been on the team forever. So I've worn a lot of hats and done a lot of roles. Natalia Godyla: So what were some of the other roles that you've been at at Microsoft? What was the first one that brought you to Microsoft?Randy Treit: I've been at Microsoft 20 years. I started in the exchange team and worked on some mobility stuff. But pretty quickly... So I started in 2000. In 2003, I joined the antivirus team, which was brand new at Microsoft. Really Microsoft's first foray into trying to get serious about the antivirus space. And I joined as a program manager, actually. So security research is a fairly new role for me, but was basically worked on the backend infrastructure for the antivirus platform in the early days. And that was the days of worms running rampant everywhere you had SQL Slammer, MSBlast, Sasser worm, Code Red, Nimda. All the greatest hits of when security was a very dark, dark time at Microsoft. And that's when I started and then have done a ton of stuff since then. So I worked on the antivirus engine as a PM and from the engineering side. Eventually moved on to do a lot of work with our cloud protection system in the last period. And then, about two years ago, I guess I moved from engineering side into security research. Natalia Godyla: So were you sold on security after being part of the AV team? Was that what did it for you?Randy Treit: Our customers, Microsoft's reputation, friends and family, everybody was just getting hammered by security threats at the time. And I really wanted to do something about that. Working on exchange was fascinating from a technical perspective, but getting into the security space where there was a real opportunity to go to battle against the bad guys and try and really protect. I'm sure we all, back in those days, this is mid-2000s, early 2000s, had friends and family who got hit by a worm or a virus or a scam. And so it was very motivating for me to get into a place where I could do something about that. And that's sort of driven me ever since. And I've done a few other forays into some stuff, like I took a break from security for about two years. Around 2012, went and worked on Xbox for the Xbox One when that was getting released and learned a ton about services. And that was a good break, but I couldn't stay away from the security space. Nic Fillingham: Randy, I'd love to come back to that first gig of yours working in the anti-malware space. So for whatever reason, I actually went down a rabbit hole recently trying to better understand the history of Defender. It sounds like you were there at its sort of inception. My understanding is that the first anti-malware, antivirus client, first of all, it wasn't built into the OS. It was a download. And was it something that we built in-house or was it an acquisition? Was it a combination? Do you know the history? Were you there for that?Randy Treit: Yeah. So I was the third PM hired into the antivirus team and it was right after the decision to acquire RAV from a Romanian company called Gecad. And so I started on a Monday and on Wednesday all of the Romanian developers showed up, many who are still on the team today. Marty Marinescu, who was the lead developer of the engine, is still the lead architect on the antivirus engine. And I remember, it was an interesting cultural experience, because they all came in and the custom in Romania was that you would, every morning, go to everybody's office and shake their hand and greet them in the morning. And so that was- Nic Fillingham: That's awesome!Randy Treit: Yeah, it was great. Unfortunately they, I think, became acclimatized to the not as polite American way of doing things. That sort of died out after a few weeks. But yeah, it was an acquisition and we didn't actually know what we were going to do with it at the time. So there was always a desire bring the protection capability into the operating system, but that's a big rock to lift and eventually we got there with Defender in the interim. It started out as, like you said, a download. So the initial... For years we've had the malicious software removal tool that comes out every patch Tuesday and runs on everybody's machine to clean up the ecosystem of malware.Randy Treit: But before that it was actually the very first release of the same engine that runs in Defender today, was something called Blast Clean. It was a Blaster removal tool to remove the Blaster worm. And we released that in late 2004. I have some stories about testing it out on my home machine and actually infecting it. And my kids not being able to play Magic School Bus the next day, and getting a call at the office. So those were fun times. Nic Fillingham: Can you elaborate on that? Is that the story? Is there more to it?Randy Treit: So what happened was the Blaster worm, there was a particular patch that if you weren't patched, it would infect your computer within a few seconds of being online. And so we had the early builds... This was December, heading into Christmas season in 2004. And I decided, well, I've got my computer at home. I'll just uninstall the patch and let it get infected. And then I will run our removal tool and make sure that it works. It was not the brightest thing to do. Don't do this at home kind of thing. I was younger and more eager to just do crazy stuff that I would probably be a little more careful these days, but I did it. I uninstalled the patch. The machine got infected. Rebooted, which was part of the infection.Randy Treit: And then it came up and I ran our removal tool and it worked great and then I decided to try it again. So for those who may remember the Blaster worm, there was another worm called Nachi that somebody else had written and released, exploiting the same vulnerability. And Nachi tried to remove Blaster and then patch your computer. And so our tool removed both of those. And what happened, in my case, was the machine got infected with Nachi, but it was a copy of the Nachi worm, that had itself been infected with a file infecting virus, which infected all the executables is on my machine and then basically bricked it and made it so it wouldn't boot. Nic Fillingham: I know that I got infected with Blaster worm. I couldn't remember that because I got in big trouble from my dad.Randy Treit: Oh, yeah. Nic Fillingham:But I sort of can't remember what it did. I know that it stopped... No one could use the computer. It just completely... The computer was unusable, but can you just kind of bring us down memory lane? If you were infected by Blaster worm, what actually happened?Randy Treit: It was not a worm that was exfiltrating data off your machine. Now it's all about money and these crime groups trying to exploit the ecosystem with Ransomware and that kind of thing. It was really just designed to spread. So it was purely, as I recall and if I'm remembering correctly, but it would just try and infect... It would infect your machine and your machine should actually be able to run with the infection. Although like in my case, and maybe in yours, if it got infected with a version that was itself infected with something else, it would just brick the machine. Like if there was a file infector, which is what I experienced with the Nachi worm. But essentially it would just try and spread to other machines that were unpatched, randomly spraying IP addresses trying to find another machine that had the vulnerability. Natalia Godyla: So you mentioned that, right now, part of your role is to focus on protecting patient zero. So how is that different than some of the work you've done in the past? And what's different about focusing on patient zero in specific?Randy Treit: The attackers could guarantee that they could release something into the wild that wasn't detected because it wasn't detected by current signatures. So before we had cloud protection, you just had the heuristics and signatures that were on disc in these virus definition updates that computers would download periodically. Typically, a few times a day. So you couldn't really protect patient zero because the attackers would always be able to tweak their malware until they saw from scanning with, say, the virus signatures that you weren't going to be able to detect it. And then they would release it. And then the clock starts ticking at that point. And you have a certain amount of time before, say, a customer reports that to Microsoft, or we discover that a sample from some sort of honeypot, or whatever.Randy Treit: And then now you have, okay, we need to quickly add a signature and ship that out to the customers. So the cloud has been a real game changer because it gives us an opportunity to run all these machine learning models in real time, in milliseconds to make an evaluation of a file that we've never seen before and decide that it's malicious and then block it. That has been a huge game changer in terms of protection capability and really shrinking that time to protection to milliseconds from where it used to take days and hours to get a signature out. Nic Fillingham:And how do you measure the false positive rate? If there is one, in that sort of protecting patient zero. How do you measure and then how do you find that balance between a couple of false positives, which would be, probably, annoying. But do you allow yourself a few of those to slip through in order to genuinely protect patient zero? Or are the models so good now that the false positives are extremely rare?Randy Treit: Oh, well, we're always going to have some false positives. ML is not perfect and human expert rules and human logic is not perfect. So there always will be false positives. We have certain thresholds that we try and keep our rules under, or that are basically lines in the sand that, hey, in order to release a new, say, detection rule in our cloud protection infrastructure, it has to run in an experimental period for a certain amount of time. Typically, even a few weeks while we gather all the data on what it would have blocked on, and then we can evaluate, is it having a nice, low, false, positive rate? So there are certain thresholds that we need to make sure all those rules are running under.Randy Treit: And then we have guard rails to make sure that if all of a sudden a rule or an ML model starts... Something changes under the hood and it starts having too high of a false positive rate, then we have systems to alert and automatically disable things until somebody goes and investigates and that kind of thing. So we're definitely very cognizant of trying to find that balance between blocking the bad stuff, but not causing too many false positives and causing pain and headache for our customers. Nic Fillingham: And does your team monitor those metrics? Is that what your team, as part of looking after patient zero, is that one of the things that you track day to day, or is that another part of the org?Randy Treit: Yeah, it's definitely our team. There are other kind of data science focused people who will do a lot of the infrastructure work to support running those metrics. But our team looks... That's creating the cloud rules and some of that capability. We'll work on writing watchdogs and guardrails and alerts and things like that. Just as part of the end to end pipeline of creating that protection. Nic Fillingham:What are some of those tools that you use day in, day out, Randy? When you start your day, where are you going to? Do have some sort of team dashboard, or are you going into some kind of Azure ML service? Yeah, what's in your toolbox?Randy Treit: So we definitely have our dashboards and tools that are the sort of go-to place for, oh, you want to see the trend of detections over time, and these kinds of things and monitor your rules and whatnot. I tend to go a lot deeper into the actual data. So I'm a big fan of Jupyter notebooks and pandas on Python. I've done a bunch of stuff in R, in the last couple of years. Lately I've been using Databricks notebooks, which are fantastic because it basically lets you do big data. Sorry. I don't know if you're familiar with the notebook type environment, but it's essentially a combination of marked down notes and graphs and visualizations. Nick, I know you've seen some of my heat maps that I like to generate, showing where we're seeing particular attacks happening globally.Randy Treit: That's all done in this notebook environment where you have this data under the hood. You can write Python code or R or Scala, and then, to process the data, and then not the other, it'll spit out a beautiful global heat map or graphs or data. And you can just sort of have instant querying at your fingertips. So typically, my day starts with usually firing up some kind of a notebook, pulling in some data. Randy Treit: I'm often looking for gaps, so where are we not doing well. So what did we see over the last... Let me find files that we're now blocking in the cloud, because our cloud learned that these are malicious, but maybe we miss patient zero and maybe we missed the first 25 encounters. Now, then we started blocking. Oh, let me figure out what happened there. Why didn't we block? How do we close that gap? Randy Treit: My day job, I would say, is really trying to find protection gaps where we're not doing a good job and figure out how we close them. They go actually implement something to close those gaps. I tend to work with Python mostly day-to-day in a Jupyter Notebook or more recently, these Databricks Notebook type environment. I love it. Compared to the old days of you're running just SQL queries against a small set of SQL data, the things you can do with these, I would say, data scientist type tools like Jupyter Notebooks is very freeing. I guess that's how I would put it. Nic Fillingham: Randy, what's flagging those gaps? So, you said you look for gaps. Is that just your experience, your expertise, you know what you're looking at when you see data, when you see dashboards, when you see reports; or are there a combination of processes that are specifically looking for a detection that picks something up and then went backwards in time and realized that "Oh, here are some historical detections that we actually miss"? How do you find gaps? I think that's the question. Randy Treit: It's a combination of manual spelunking on into the data and going off intuition or things I've done before, but we do have automation that will flag certain events. We have watchdogs and other rules that researchers write. In my mail inbox in the morning, often, I will have a list of these potential misses where maybe we missed detection on the first patient 0 through 10, and then we started blocking. So, I might go and look at, "Oh, let me dig into that a little bit and find out what happened there." So, in some cases, it might be that we have a malware probability threshold that we were looking forward to say from an ML model that says, "Oh, block if the probability is 0.95. So, 95% probability that this file is malware." Randy Treit: Going into the data in telemetry, I might see that we didn't block because the probability was 0.93. So, one of the things I would look into then, oh, can we reduce that probability that we're looking forward to block from that 0.95 threshold to 0.93? Maybe code up something to model that or to run for a few days in experimental audit mode and see, "Does that lower threshold still meet our false positive targets?" If that's looking good, we can turn that on live, something like that. Natalia Godyla: This is a bit of a deviation, but it would be great to understand, "What kind of context do you bring to this role from previous jobs? What were you studying in school? What did you intend to do? What were your jobs prior to Microsoft, and how do you use them in your day-to-day?" Randy Treit: Yeah, that's a great question. So, I was actually studying Philosophy in Pacific Lutheran University down in Tacoma. I'm a native Washingtonian. So, Microsoft was right in my backyard. It was basically the height of the dotcom boom and the end of the '90s. I had finished up the Philosophy Program at PLU and was planning to become a philosophy professor but needed to get a job. In the interim, I was married. We had a young child, another one on the way. So, I decided to take a break from school, get a job. I started as a technical writer actually at Microsoft on the Exchange Team. I think you talked to Emily Hacker. I listened to the interview and learned that she was also started as a technical writer. So, that was pretty cool. Randy Treit: And then worked in exchange for a few years before I got asked about joining this newly formed antivirus team. I made the jump there. I actually never finished my four-year degree. So, I made a plan with my advisor. I finished the philosophy program, but still had some general university stuff to finish up. But once I started at Microsoft, I was just off and running and never looked back. So, it's been an interesting journey. Sometimes I definitely suffer from, I would say, imposter syndrome here and there, where I spent a lot of time writing code day-to-day, but I've never been formally trained in computer science. It's all been self-taught or picked up on the job thing. Randy Treit: When I moved from a program management and the engineering side into research, I came without the deep reverse engineering background that a lot of my colleagues had. So, that was something that I felt like, "Oh, this is going to be hard for me to pick up." Sometimes that lack of a formal academic background, I feel like it was a bit of a chip on my shoulder, but I just try and do the best I can and go from there. Nic Fillingham: Talk a bit about philosophy, and then I'd love for you to talk about how and if you use it in your job today. Randy Treit:Yeah. So, I was not a good student in high school. So, I barely graduated high school with a very low GPA. So, when I decided to finally get my act together and go back to school, I started at a community college. I needed to take English 101 just as part of every college requirement. So, the English 101 class I took was a combined English 101 and Philosophy taught by two professors who were husband and wife. Debbie Kuder, the wife taught the English portion, and then her husband, John taught the Philosophy portion. It was basically an amazing class. My identical twin brother, who also works at Microsoft by the way, was in the same class with me. We both just fell in love with philosophy. Randy Treit: I think, I just love the idea of open-ended questions that had no answers. So, philosophy, I think differentiated from the sciences, it's dealing with questions that will never actually be answered, like what is beauty and what is a good argument? There's always going to be different opinions. Just the idea of these big open-ended unsolvable questions, but the people will keep getting closer and closer to the truth hopefully over time, I just fell in love with that. In terms of applying philosophy at work, I think the biggest thing that I got out of studying philosophy in undergraduate school at PLU was the rigorous approach to problem solving. So, even though you have these big open-ended problems, like I said, there probably are never going to get answered. Randy Treit: The approach of philosophical approach is very rigorous and requires incredibly good communication skills to be able to communicate your ideas effectively and, in your arguments, cogently. That, I think, has stood me in extremely good stead in my career. I think that's one of the things that I bring to the table. I think someone like Emily, you mentioned with the journalism background, it's just that ability to communicate. There's so many brilliant people who work in the technical field, but who are unfortunately not great communicators. Often, they need someone to help translate what their brilliant ideas into something that other people can actually understand what they're aiming at. Randy Treit: That's something that I think I've been able to do fairly successfully. Just that ability to really rigorously attack a problem and break it down into small components, which I think comes from some of that training I think has also done a great job or has stood being a good stead with malware analysis and threat analysis and that kind of thing. Natalia Godyla: So, I know Nic is dying for me to ask this, but you said you had an identical twin, you just dropped it in there casually that works at Microsoft. Do you guys pull pranks together. Have you done it as kids? Do you do it at Microsoft? Randy Treit: You have no idea. So, Mike actually worked on the antivirus team at the same time as I did. So, he joined Microsoft before me and has worked on NT 5, which became Windows 2000 and is a brilliant dev, but he was actually one of my devs and I was his PM working on the antivirus. This is probably mid-2000s. For a number of years, we were on the same team. And then he went off to Intune. But I mean, the amount of confusion we caused when people would walk into meetings or even just down the hall, it was quite fun. I'm sure we played some pranks. It's been great. Randy Treit: There was one time very early on, we weren't on the same team at that point, where he was in my office over an exchange. He had come over to grab a coffee. He was across the street. I had gone down to get a refill or use the restroom or something. This guy, David came in and started talking to Mike, like he was me, "Hey, Randy, I've got some questions about this thing." Mike was like, "Oh, I'm not Randy." David looked at him and just shook his head and said, "So, anyway, I've got questions. Do you know about this?" Mike's like, "No, I'm not Randy." He looked at him and he said, "wait, are you serious?" So, we've had those kinds of incidents. Randy Treit: Mike is my go-to person whenever I get stuck on a programming problem, because he's a brilliant programmer. So, I'm constantly sending him my code and saying, "Hey, I'm struggling with this." He usually responds with something like, "What is this monstrosity?", and things like that since I'm not nearly the coder that he is. Natalia Godyla: Subtle. Nic Fillingham: Who's the older twin by a fraction of a second or a minute?Randy Treit: Mike's four minutes older than I am. Nic Fillingham: I love it that your prank was actually a wholesome misunderstanding, an unintentional wholesome misunderstanding. I was typing frantically with Natalia, trying to see if there was some example, where you each went to the other's annual review and just tried to just say ludicrous things to the manager to see when they caught on, but no.Randy Treit: No, I haven't done too much of that at work. Although, I mean, in high school, he would skip class and I would go to his art class, because I had a girlfriend who was in the same class. One day, I got called up to make a presentation, the person they thought I was Mike. I was completely unprepared and I just fumbled my way through it. I learned that, "Oh, that didn't work out the way I was hoping it would." I'll throw this out there. My younger brother also works at Microsoft. He is a producer on Xbox video stuff. So, there's a bunch of us running around. Nic Fillingham: How many other Treits are there?Randy Treit: My sister, Tammy worked on Exchange at the same time I did back in the day. There are six of us Treit siblings. I guess four of us have worked at Microsoft. My younger sister is a doctor in Seattle, and my older sister is a teacher in Germany. Natalia Godyla: Thank you, Randy. We're happy to have you at Microsoft. Happy to have two-thirds of your family at Microsoft here, and we'll definitely love to have you back. Randy Treit: That was a lot of fun. I really enjoyed the conversation. Natalia Godyla:Well, we had a great time unlocking insights into security from research to artificial intelligence. Keep an eye out for our next episode. Nic Fillingham: Don't forget to tweet us, @msftsecurity, or email us at with topics you'd like to hear on a future episode. Until then, stay safe. Natalia Godyla: Stay secure. See for privacy and opt-out information.
Yeehaw! “Data Cowboy” is in the building. Join us as Nic Fillingham and Natalia Godyla sit down with Ram Shankar Siva Kumar, aka “Data Cowboy” at Microsoft, for an exciting conversation about the release of a new adversarial ML threat matrix created for security analysts. Have no fear, we made sure to find out how Ram acquired the name, “Data Cowboy”, so saddle up and get ready for the ride!Stick around to hear Nic and Natalia explore the urgency of surfacing threats at a faster rate with Justin Carroll, a Threat Analyst at Microsoft, and why it is more important now than ever before.In This Episode, You Will Learn:  • How Microsoft is using the new ML threat matrix against cyber attacks • The approach and philosophy for putting the threat matrix on GitHub  • ML applications in regard to healthcare and why it is worrisome • What needs to happen in order to be successful in combating certain threats   Some Questions We Ask: • What is an adversarial ML threat matrix? • How will the community on GitHub contribute to the evolution of the ML threat matrix? • What resources are available to learn about all things VM? • What techniques are being used to find threats at a faster speed? • How do AI and ML factor into the role of managing data and collaborating with other teams? Resources Ram’s Blog: Microsoft Security Blog: Nic’s LinkedIn Natalia’s LinkedIn  Ram’s LinkedIn Justin’s LinkedIn  Transcript(Full transcript can be found at Fillingham:Hello, and welcome to Security Unlocked. A new podcast from Microsoft, where we unlock insights from the latest in news and research from across Microsoft security engineering and operations teams. I'm Nic Fillingham.Natalia Godyla:And I'm Natalia Godyla. In each episode, we'll discuss the latest stories from Microsoft security, deep dive into the newest threat intel, research and data science.Nic Fillingham:And profile some of the fascinating people working on artificial intelligence in Microsoft security. If you enjoy the podcast, have a request for a topic you'd like covered, or have some feedback on how we can make the podcast better.Natalia Godyla:Please contact us at or via Microsoft security on Twitter. We'd love to hear from you. Hi Nic. Welcome back. How were your holidays?Nic Fillingham:Yes. Thank you, Natalia. Welcome back to you as well. Mine were great. You know, normally you drive somewhere or you fly somewhere, you go visit people, but this was all the FaceTimes and the Zooms and the Skypes, staycation, but it was still nice to eat too much and drink too much over the holiday period. How about you?Natalia Godyla:Yes, it was... to quote my boss. "It was vegetative." It was definitely just... well actually you know what? I did have a big moment over the holidays. I got engaged. Nic Fillingham:Oh, what! Natalia Godyla:I know.Nic Fillingham:Congratulations. Natalia Godyla:Thanks.Nic Fillingham:That's amazing. Natalia Godyla:I feel like it was absolute relaxation, really high point during the five minute proposal. And then we went back to our natural state and just absolute relaxation, lots of video games.Nic Fillingham:Hang on. So were you both sitting on the couch, playing some switch, eating your 95th packet of Doritos, and then all of a sudden your partner pauses and says, "You want to get hitched?"Natalia Godyla:There was a little bit more pomp and circumstance to it. Though I think that would have been very fitting for us. Nic Fillingham:Wow! Good on you guys. That's awesome. Natalia Godyla:I'm sure that like us, everyone has forgotten what they were doing at work, and I'm sure also what this podcast is doing. So why don't we give everyone a after the holiday refresher?Nic Fillingham:So just before the holidays, we partnered with Petri who run the site First Ring Daily, a bunch of other great blogs, podcasts, email newsletters, and so welcome to all our new listeners who've come to us from Petri, from Throughout from First Ring Daily. Yeah. So what is security unlocked? Well, first and foremost, Natalia, and all your co-hosts, we are Microsoft employees and we will be interviewing, and we do interview on this podcast, other Microsoft employees, but we talk about security topics that hopefully are relevant to all security professionals and those who are interested in the state of cybersecurity. Nic Fillingham:And what we'll do in each episode is the first half is we'll pick a sort of a recent ish topic and we'll speak to a subject matter expert or an author of a recent blog post and ask them about the thing that they're working on, or that they've announced in the AI and ML space, hopefully try and demystify some new terms or concepts that may be either nascent or sort of difficult to wrap one's head around. And then in the second half...Natalia Godyla:We talk to again, another Microsoft security expert, this time more focused on the individual and their path to cybersecurity. So we'll ask them about what interested them about cyber security, what compelled them to join the industry, what jobs they've had, how they've come to Microsoft or their current role. In addition, we also have a new announcement about the podcast, which is we'll be switching to a weekly cadence. So prior to this, we were bi-weekly, now more goodness coming your way.Nic Fillingham:More pod in your pod app. What is the collective receptacle for pod? What is it? More pods in your cast, more cast in your pod?Natalia Godyla:More beans in your pod.Nic Fillingham:I like that. More beans in your pod. And I think the other thing that's worth reiterating Natalia is if you have a cyber-security topic you would love to learn more about, or a perspective you'd like to hear from, please let us know, we'll go after it for you and try and bring that to a future episode.Natalia Godyla:Yes, absolutely. We're really thankful to everyone who has reached out thus far and just keep it coming.Nic Fillingham:On today's episode in the first segment, which we call our deep dive, we speak with Ram Shankar Siva Kumar, whose title I will not give away in the intro because we talk about it in the conversation. And it's an awesome one. Ram works in the Azure Trustworthy ML team. And he's here to talk to us about a blog post that Ram co-authored with Ann Johnson that announces a new adversarial ML threat matrix that has been built and published up on GitHub as a collaboration between Microsoft, MITRE, IBM, Nvidia, Bosch, a bunch of other organizations as a sort of open source approach to this upcoming sort of nascent threat category in adversarial machine learning. And it was a great conversation. And then after that, we speak with...Natalia Godyla:Justin Carroll of the Microsoft Threat Intelligence Global Engagement and Response team. He started in networking very on the ground and only got his education in cybersecurity later in his career, which I think to anybody out there, who's looking to transition to security, who has a different background in security and is wondering whether they can make it, you can. He also chats a little bit about what inspired him to join cybersecurity. Some of it came from video games, which is a theme we're seeing again and again.Natalia Godyla:So he had a unique spin on vigilantism within video games and ensuring that those who had an unfair advantage by using mods were checked and tried to level the playing field for all the rest of the players of that game. And of course we touch on Ninja Turtles, which is really the highlight of the episode. I think, with that on with the pod.Nic Fillingham:Ram Shankar Siva Kumar, thank you for joining us on Security Unlocked.Ram Shankar Siva Kumar:Hey, thanks for having me, Nick and Natalia. Really appreciate it.Nic Fillingham:So we're going to talk about a blog post that you co-authored with the wonderful Ann Johnson. The title is, it's a great title. I'll get straight to the point. Cyber attacks against machine learning systems are more common than you think. Before we get into that, though, I just have to ask, you list your title as data cowboy, which is fantastic. I would love data cowboy, anything cowboy. I would love that for my title. Could you explain to people, what does a data cowboy do and what is the Azure Trustworthy ML group?Ram Shankar Siva Kumar:Oh, totally. First of all, this is like every kid's dream is to be Woody from Toy Story. It's just like, I realize it in my own way. So when I joined Microsoft in 2013, there really wasn't an ML engineer position. So my boss was like, "You can be whatever you want. You can pick your own title." I was like, "Yes, Toy Story comes to life." So it was like, this is a brown version of this Woody that you kind of get. So basically what the Trustworthy Machine Learning group does is our promise to Microsoft is to essentially ensure we can enable engineers and customers to develop and deploy ML systems securely. So it's kind of a broad promise that we make to Microsoft and our customers.Nic Fillingham:Got it. I would love to come back to just the data cowboy one more time. Tell me what you do. I mean, I have visions of you riding around the office on a hobby horse. Lassoing errant databases. Tell us about your day to day. What does it look like?Ram Shankar Siva Kumar:Yeah. So what really happens is that, like I said, I really wish I can ride it on my office, now I am at my home and my 500 square foot apartment- definitely not recommended. But most of the time we end up doing is this wonderful Hiram Anderson who's part of our team, he's militantly looking at how we can detect attacks on machine learning systems. So really working with him and the rest of the Microsoft community to kind of keep our eyes and ears on the ground, see like what sort of attacks on machine learning systems we are seeing, our various different channels and trying to see how we can detect and respond and remediate those sort of attacks. So that's the first one big one. The second thing is like I get to work with a wonderful Will Pears. So I get to work with him to think about actively attacking red teaming Microsoft's machine learning system. So even before our attackers can look at, exploit the vulnerabilities Will and Hiram go and actively attack Microsoft ML systems.Natalia Godyla:So how does the work you do connect to the different product groups. So as you're identifying these cyber attacks, are you then partnering with our products to build those into the detections?Ram Shankar Siva Kumar:Yeah, that's a great question. So one of the things I really like about Microsoft is that super low slake to meet with somebody from another product team. So the amazing Mira Lane who heads the Azure Cognitive Services, really worked very closely with her. And I believe you ever had a Holly Stewart in your podcast as well, so worked very closely with her team. So it's really a big partnership with working with leaders from across Microsoft and kind of shopping around what we're doing and seeing how we can kind of help them and also learn from them because they also have sensors that necessarily might not have.Nic Fillingham:Let's talk about this blog post. So you and Ann both announced this really interesting sort of consortium of 11 organizations, and you're releasing an adversarial ML threat matrix. It's open source, it's on GitHub. Very exciting. Tell us about it.Ram Shankar Siva Kumar:So the goal of the adversarial ML threat matrix is essentially to empower the security analyst community so that they can start thinking about building detections and updating their response playbooks in the context of protecting ML systems. And one of the things that's kind of like we want to be mindfully different is the attacks that we see to this framework with, all these techniques, we kind of only put the ones that Microsoft and MITRE jointly vetted that were effective to be against production machine learning systems. Ram Shankar Siva Kumar:So first of all, the whole area of attacking machine learning systems goes all the way back to 2004. In fact, you can find Daniel Loud, whose Twitter handle is Dloud on Twitter today. He continues to work on this super cool fields and there's a wonderful timeline by this other researcher called Battista Bisho that he also linked to the blog, but he can basically see that this work has gotten immense academic interests for the last 16 years. And especially in the last four years after a very seminal paper was released in 2014.Ram Shankar Siva Kumar:So when a lot of people think about spiel, they think of as, oh, this is something that is really theoretical. This is something that... Oh, Great, you're working in academic setting, but no, that's not true. There are marquee companies, who've all had their ML systems subverted for fun and profit. So the whole point of this blog post with MITRE and this whole corpus of industry organizations was, this is real. Attacks on machine learning systems is real, you need to start thinking about this.Ram Shankar Siva Kumar:Gartner released a report on 2019 saying, 30% of all cyber attacks in 2022 is going to involve a tax on machine learning systems. So this is not a pie in the sky. Oh, I'll get to it when I get to it. 2022 was a year and a half, it's a year away from now. So we got together in this blog post to really empower our security analysts community and help them orient for this new threats.Natalia Godyla:Can you talk a little bit more about what exactly is the adversarial ML threat matrix and how you envision security analysts using this tool?Ram Shankar Siva Kumar:Yeah, totally. So one of the things that before we even put this matrix together, we kind of conducted a survey of 28 organizations. We spoke to everybody from SMBs to governments to large organizations and we spoke to the security analyst Persona, as well as the MLG person. I asked them, "Hey, how do you think about securing ML systems? This is a big deal. What are you doing about it?" And they were like, "Well, we don't have the tools and processes in place to actually go and fix these problems." So the first thing we realized is that we wanted the security analysts community to be introduced to adversarial ML as a field, try to condense the work that's happening in a framework that they already know. Because the last thing we want to do is to put another framework another toolkit on their head.Ram Shankar Siva Kumar:And they're just going to be like, "Nope, this is not going to work out. This is one more thing for them to learn." So we took the MITRE's attack framework. So this is something that was again, bread and butter for any security analyst today. So we took the attack framework and we kind of said, "Hey, we've been really cool." If you took all the ML attacks and put it in this framework, and that's exactly what we did. So if you look at our track matrix, it's modeled after the MITRE attack framework. Ram Shankar Siva Kumar:So the wonderful folks from MITRE's ML research team and us, we got together and we basically aligned the attacks on machine learning systems, along reconnaissance persistence, model evasion, ex-filtration. So if you look at the top of our matrix, the column headers are essentially tactics and the individual ones are techniques.Ram Shankar Siva Kumar:So let's say that an attacker wants to gain initial access to a machine learning subsystem, let's say that's her goal. So she has a couple of options to kind of execute her goal. She has a couple of techniques in her kit. The first thing is that she can just send a phishing email to an ML engineer. That's very valid. Phishing is not going to go away. The second thing that she can do is she can take a pre-trained ML model available that people generally download and she can backdoor it. So the whole point of this attack matrix is to A, build a common corpus of attack techniques and attack tactics in a framework that a security analyst already has knowledge of.Natalia Godyla:Are you seeing any trends? What's most common to combine.Ram Shankar Siva Kumar:Oh, that's a great question. So before I just step into this, I first want to tell you about this attack called model replication. So the easy way to think about this and Natalia, I will get to this, I promise. Natalia Godyla:I love the excitement. I'm so ready for it.Ram Shankar Siva Kumar:We're going to take a little detour like Virgil and Homer. So essentially the best way to think about model replication is that open AI is a very famous ML start up. And they last year released a model called GPT-2, and they said, "Hey, you know what? We're not going to release the entire model immediately. We're going to release it in a stage process." We're going to just... because we want to do our own verification and before they could release the entire model, these spunky researchers, so I love that. They're still cool. Vania Cohen. And I know this other person's name is Skylion with a O, they replicated GPT-2 it was like 1.5 billion parameter model, and they've leased it on the internet on Twitter. And they call it open GPT-2. And I love their tagline, which is GPT-2 of equal or lower value.Ram Shankar Siva Kumar:So even before the company could release, they replicated the ML model based on the data sets that were available based on the architecture. And they basically at the end of the day, and we also references our case study is that they basically tweaked an existing model to match GPT-2 and they publish that for everybody to use. No, it does not have the same accuracy or the same metrics as the original GPT-2 model. But the fact that an attacker can even replicate a ML model using publicly available data sets and having some insights about the architecture is something for people to think about.Ram Shankar Siva Kumar:So now to come back to your excellent question. So what exactly is a common pattern? So what essentially we see attackers doing is that they go interact with the machine learning system, attackers might send some data. They might get some responses back and they keep doing that enough amount of time. And they now have sufficient data to replicate the ML model. So the first step is that they go and replicate the ML model and from the ML model that they have replicated, they go do an offline attack. Because now they their own ML model, they try to evade this ML model and then they find a way to evade the ML model. And they take the examples of the test points that evade the ML model and now evade the online, the real ML that's out there taking that and then boom, fooling the real online ML model. So that's a common data point, but three case studies in our adversarial ML GitHub page that actually kind of shows this.Nic Fillingham:So the sort of takeaway from that. If your data set is public, don't make your ML architecture public and or vice versa.Ram Shankar Siva Kumar:That's a great question. And I've been thinking about this a lot, first of all, we definitely want to be transparent about the baby builder ML models, right? Marcus Sanovich, Oh gosh, he's such an amazing guy. But for the last so many years in RSA has been like militantly, been talking about how we build our ML models for security purposes, because we want to give insights into our customers about how we actually built ML models. And the data sets are machine learning as a field, it has as norms of opening up our data sets. In fact, one can attribute the entire deep learning revolution to Dr. Fei-Fei Li's image in a dataset which really sparked this whole revolution. So, I really don't want anybody to think that being open with our data sets or being open with our ML platforms is a good idea.Ram Shankar Siva Kumar:Because even if you think of traditional cyber security, right? Security by obscurity is never a good strategy. So the way we want to push people to think about is how are you thinking about detection? How are you thinking about response? How are we thinking about remediation? So really trying to take the assumed breach mindset and feeding it into your ML systems is how we want to push the field towards. So if you take away anything from this is continue to be opening your systems for scrutiny, because that's the right thing to do, that's the norms that we've set. And that's important to advance research in this field and think about detection strategies and think about, and assume breach strategies for building ML systems. Ram Shankar Siva Kumar:We wanted to distinguish between traditional attacks and attacks on ML systems. So the one thing that I want to think about is the threat matrix contains both traditional attacks and attacks on ML systems. Whereas the taxonomy only contains attacks on ML systems. The second difference is that, like I said, the matrix is meant for security analysts. This one is meant for policymakers and engineers. The third that's the more important difference is that in the context of the threat matrix, essentially we are only putting attacks that we have validated against commercial ML systems. It's not a laundry list of attacks. We're not trying to taxonomize. Nic Fillingham:I wonder if you could talk about the approach and the philosophy here for putting this on GitHub and making it open to the community. How do you hope folks will contribute? How would you like them to contribute? Ram Shankar Siva Kumar:Yeah, absolutely. So Miguel Rodriguez, who runs the MITRE, who we collaborated with, wonderful team over there before putting this out on GitHub, there was a little bot of angst, right? Because this is not fully baked product. This is something that 13 organizations found useful, but doesn't mean everybody in the community might find useful. And I think he said something to the effect of-Nic Fillingham:It's almost as if you're a cowboy.Ram Shankar Siva Kumar:Yeah. There you go, herding people. It was like, we're putting this out, acknowledging this is a first cut attempt. This is a living document. This is something that we have found useful as 13 organizations, but we really are hoping to get feedback from the community. So if you're listening to this podcast and you're excited about this, please come and contribute to this matrix. If you think there are attacks that are missing, if you would like to spotlight a case study on a commercial ML system, we are super looking to get feedback on this. Ram Shankar Siva Kumar:And we also kind of realized that we wanted a safe space almost to talk about attacks on ML systems. So we were like, you know what? We're just going to have a little Google groups. And the membership of the Google groups is extremely diverse. You've got philosophers that are interested in adversarial machine learning. We've got people who are looking from various perspectives, joining our Google groups and kind of like giving us feedback and how we can make it better.Natalia Godyla:Yeah. As you mentioned, there are tons of different perspectives coming into play here. So how do you envision the different roles within the community interacting? What do you think needs to happen for us to be successful in combating these threats?Ram Shankar Siva Kumar:Yeah. This is a great question. The one thing that I've learned is that this topic is immensely complex. It's mind boggling to wrap the different personas here. So I'll just give you a rundown, right? So, so far we know that policymakers are interested in securing ML systems because every national AI strategy out there is like, securing ML systems is top priority for them. ML engineers are thinking about this, academic researchers. There were like 2000 papers published in the last, I want to say five or six years on this topic. So they are like a hotbed of research we want to rope into. We've got security analysts from these companies that we're talking to are interested. Csos are also thinking about this because this is a new threat for them. So as a business decision maker, how should they think about this?Ram Shankar Siva Kumar:One thing that I got an opportunity with Frank Nagle, who's a professor at HBS. We wrote up piece at Harvard Business Review talking about, is it time to insure ML systems. ML systems are failing so if you're ML powered like vacuum cleaner burns a home down, what do you do about it? We try and rope in the insurers to come in participate in this. So, Natalia this is such a green field and the only way we're going to like get ahead to really get people excited and try for clarity together as a community.Nic Fillingham:How would an ML powered vacuum cleaner work?Natalia Godyla:I was going to say that sounds like a 2020 headline, ML powered vacuum cleaner burns down house and threat.Ram Shankar Siva Kumar:Oh my gosh. So, okay-Nic Fillingham:Man bites dog. Ram Shankar Siva Kumar:There you go. It's funny because this was not an example that I made up. I wish I did. I know. Yes, Nic. I see, yes.Nic Fillingham:What? Ram Shankar Siva Kumar:Yes. Nic Fillingham:All right.Ram Shankar Siva Kumar:This is a well-documented paper called a concrete problems in AI safety. And they talked to the most it's like Final Fantasy. Everything that needs to go wrong is going wrong. So, they're like robots that are burning down homes, breaking things that they can clean up. So if your machine learning system is not trustworthy, there are going to be problems. And you really need to think about that. Nic Fillingham:I can't even get my kettle to boil.Ram Shankar Siva Kumar:But the thing that really worries me is ML applications used in health care. You keep seeing headlines like machine learning systems being used by radiologists, amidst radiologists when it comes to identifying Mulligan tumors and things like that. There's a fantastic work by Samuel Finlayson from Harvard. He show that if you take an x-ray image, just take it and slightly rotate it and you give it to the ML system. It goes from very confidently thinking that it's malignant to very confidently judging it's benign. And that is really scary.Ram Shankar Siva Kumar:In the beginning of the podcast, we spoke a lot about how an adversary can subvert machine learning systems for fun and profit. Oh boy, there is an entirely separate world of how machine learning systems can fail by themselves. What we call unintentional failure modes. And trust me, you will want to go live in the middle of the North cascades in a cabin after you read that work. It'd be like, I am not getting anything ML powered until they figure this out. But the good news is there're extremely smart people, including Hiram and Will from my team who are looking into this problem. So you can feel a little bit like a shore that they're the true Avengers out there.Natalia Godyla:I love all the head nods from Nic. I feel like it underscores the fact that we only know a percentage of the knowledge on ML. So we just need a community behind this. No one company person can know all of it. Ram Shankar Siva Kumar:Absolutely. Oh my gosh. Yeah. When we open the adversarial ML threat matrix Google group, we now went from zero. We felt like nobody's going to join this Google group. It's going to be like a pity party where I'm going to email Michel from MITRE and he's going to respond back to me. But no, we went from zero to 150 right now over just the last four days.Natalia Godyla:Ram, thank you for giving us all of this context on the adversarial ML threat matrix. So what's Microsoft's continued role. What's next for you in ML?Ram Shankar Siva Kumar:First of all, we are hiring. So, if you'd like to come and join us, we are looking for developers to come and join us in this quest. So please email anybody, even Nic, and he can forward his resume. Nic Fillingham:Do you need to have a cowboy hat? Is a cowboy hat a necessity?Ram Shankar Siva Kumar:Not at all. We will accept you for who you are. Natalia Godyla:Do you provide the cowboy hats?Ram Shankar Siva Kumar:We will provide everything. Anything to make you feel comfortable. So we are growing and we'd love to work with the folks. With the adversarial ML threat matrix, like I said, we really are looking for feedback from the community. We really think that like Natalia very correctly pointed out this is a problem so big that we can only solve it if we all come together. So please go to our GitHub link. I'm sure Nic and Natalia might put the link to it. We'd love to get their feedback.Ram Shankar Siva Kumar:The second thing is if you kind of are... We are especially looking for people to come in at case studies, if you think we're missing a tactic, or if you think that you've seen an attack on a ML system on a commercial Ml system, please reach out to us and we'd be happy to include that in the repository. Nic Fillingham:If your autonomous vacuum cleaner has attempted to undermine democracy, let us know. Ram Shankar Siva Kumar:And the one thing that I want everybody to take away is that when we did our survey, 25 out of 28 organizations did not have tools and processes to kind of secure the ML systems. So if you're listening to this podcast and you're like, "Oh my gosh, I don't have a guidance." Do not feel alarmed. You're tracking with the majority of the industry. In fact, three organizations, all of whom were large in our survey even thought about this problem. So there are tools for you and processes that we put out. So in our docs at, there's a chat modeling guidance, there's taxonomy, there's a bug bar that you can give to your incident responders so that they can track bugs. And for the security analysts community, there is the adversarial ML chat matrix. So please go read them and please give us feedback because we really want to grow.Natalia Godyla:I love it. Thank you for that. That's a great message to end on.Ram Shankar Siva Kumar:Awesome. Thank you, Nic and Natalia for having me. Really appreciate it. This was really fun.Natalia Godyla:And now let's meet an expert in the Microsoft security team to learn more about the diverse backgrounds and experiences of the humans, creating AI and tech at Microsoft. Today, we're joined by Justin Carroll, threat analyst on the Microsoft threat intelligence, global engagement and response team. Well thank you for joining us, Justin.Justin Carroll:Thanks for having me. Natalia Godyla:Well can we kick things off by you just sharing your role at Microsoft. What does your day to day look like?Justin Carroll:So my role is related to threat hunting across large data sets to find advanced adversaries and understand what they're doing. Look for detection opportunities and communicate out the behaviors of the specific threats that we're finding to partner teams or to our customers to help them understand the threat landscape and kind of staying on top of what attackers are doing.Natalia Godyla:That's super interesting. And can you talk a little bit about any recent patterns that you've identified or interesting findings in your last six, eight months?Justin Carroll:Well, it's been a busy six or eight months, I would say, because everybody's been very busy with COVID. We've been seeing quite a large increase in human-operated ransomware and stuff like that. So I've been working really hard to try and figure out different ways to try and surface their behaviors as early as we can to customers to help them take action before the ransom happens. And we've been seeing quite a few other different really advanced adversaries compromising networks. Justin Carroll:A lot of it's kind of the same old, same old, just more of it, but it's always interesting and there's never a shortage of new findings each day and kind of moments of, "Oh, that looks like this, or they're doing this now." Awesome. Great.Natalia Godyla:You mentioned you're constantly trying to find new ways to identify these faster. What are the techniques that you're trying to use to find the threats quicker?Justin Carroll:There's a whole bunch of different ways that you kind of try and surface the threats quicker. Some of it's research and reading other people's work and blogs and stuff like that. I tend to live in the data most of all, where I'm constantly looking at existing attacks and then trying to find similar related behaviors or payloads or infrastructure and pivoting on those to try and attempt to find the attack, to be ready to find it as early as possible. And what's called the kill chain.Justin Carroll:So from the time that the attacker gets in the network, how quick can we find them before they've had a chance to conduct their next set of actions? So whether if they're stealing credentials or something like that, can we surface them before they've had a chance to do the credential theft and then kind of always trying to move earlier and earlier in the kill chain to understand how they got there. And then what are some of the first things that they did when they did get there and how do we surface those next?Justin Carroll:Because a lot of those are a little bit more difficult to surface because it can kind of tend to blend in with a lot of other legitimate activities. Nic Fillingham:What kind of tools do you use Justin? Are you in network logs and sort of writing queries, is there a big giant futuristic dashboard that you sit in front of and you have virtual reality gloves moving big jumps of numbers left and right. Well, what are the tools of your trade? Justin Carroll:So one of the tools that we use a lot, there is a bunch of data that's stored... Customer facing, it's usually called Azure data Lake. It's these huge databases with large amounts of information where you can construct queries with what's called KQL, I believe it's Kusto query language. So there's a specific tool for kind of deep diving into all of that data across our many different sources. And then using that to basically structure and create different queries or methods of finding interesting data and then kind of pivoting on that data. Justin Carroll:Then in addition, I've built some of my own tools to kind of help improve my efficiency or automate some of the stuff that I have to do all the time and then just to make me faster at hunting for the things that I'm looking for.Nic Fillingham:Is it an AI version of yourself? Is it a virtual Justin?Justin Carroll:No. We work with the ML team to try and share as much knowledge with them as possible. There is no tool for an AI Justin, as of yet.Nic Fillingham:Well, let's back it up a bit. So one of the things we would like to do in these interviews with the security SMEs, I'm not even sure if we've explained what an SME yet. We call it a Subject Matter Expert. That's an acronym. We use a lot here at Microsoft. I think it's pretty broadly known, but if you've heard of SME or SME, that's what it means.Nic Fillingham:Now, you and I, we crossed paths about a year ago for the first time when Jessica Payne, who actually hasn't been on the podcast yet, Jessica introduced me to you and she said, "You have to talk to Justin." And she gave me three sort of very disparate, but intriguing bits of data about you. She said, "Justin used to climb telegraph poles. He is a big Star Wars fan and is in a metal band." And I'm sure I've gotten those three things slightly wrong. Could you kind of talk about your journey into the security space and then sort of how you found yourself working for Microsoft. But first of all, these three things that Jessica told me are any of them true?Justin Carroll:Mostly they are. So some of these will kind of combine for the telephone climbing aspect. I used to work for a wireless internet provider that had leases or specific towers, cell phone towers or other towers on top of mountains, essentially, where we would have wireless radio dishes that would communicate to each other. So I was occasionally tasked with installing and or fixing said towers, which is okay if you are fine with heights, I wasn't at first, but you just kind of get used to it. And you kind of realize once you're above 20 feet, it really doesn't make any difference. If you fall, it's going to hurt, but climbing a tower in the winter and in the wind and where you can barely feel your hands and all that wasn't great. Justin Carroll:I was a pretty big Star Wars fan growing up as a kid, even more of a Ninja Turtle fan. And as for metal, I used to be in a band with some friends and have been playing guitar for 25 or 26 years. And music has been a very huge part of my life and remains to be.Nic Fillingham:I think we'll circle back to Ninja Turtles. I'm not going to let that one go, but so let's talk about your path into security. So was this you're working for the wireless internet provider was this your first job. Was this mid career. Where does that fit in your sort of LinkedIn chronology? And at what point did you use formerly into insecurity?Justin Carroll:So it's been a long and winding road to get here I would say. So the internet provider was what I would guess I'd call my first career job of sorts. I had started there in my early 20s and worked for them for about... sorry my cat is right in front of the microphone. One second. Nic Fillingham:There's a cat there. Justin Carroll:She wanted to say her piece. So I worked for the internet company for just under a decade. I used to do some networking type fun stuff in Halo 2, to kind of maybe garner a little bit of an advantage, I guess I would say, and use those learned skills to land that first job. And I did that for quite a while, but realized I was kind of stuck in this job. It was in a city that I didn't want to live in. And I had kind of maxed out my capabilities there. I had attempted to move to Portland because I wanted to have a bigger city experience. I applied to 254 jobs, got one interview for basically an office tech support role was the only position I got hired, but it wasn't feasible to live in Portland.Justin Carroll:So after quite a bit of soul searching and realizing that basically nobody cared that I had eight years of on the job experience because I didn't have a college degree. There were not any doors open for me for the most part. I then decided to take a pay cut and go get a job at a university that was just a city over and work full-time and go to school for a degree in cybersecurity while working full-time for the university doing kind of technical work for them, helping them understand their... Sorry, my cat is a whole thing right now.Nic Fillingham:Your cat's just trying to interject with like don't. Hey, you glossed over that Halo 2 thing, you better to come back to that.Justin Carroll:Aria, come here. Nic Fillingham:We're leaving all this in, by the way. Natalia Godyla:Yeah. We're very much enjoying it.Justin Carroll:So kind of advising the university on different technologies that they could use for their students. So I did that for about three and a half years while going to school and then graduated top of my class and applied for another 150 some odd jobs and mostly the Seattle area this time and was about to give up because even though I now had a degree and almost 10 years of experience, it still wasn't enough. And everybody that I kept losing to had between 10 and 20 years experience. And it just wasn't an option for folks with less specific cybersecurity experience to kind of enter the field. Justin Carroll:There were a lot of walls that were put up. I had a friend of a friend who worked for cybersecurity at a company somewhere in Arizona, who I'd never met. And he decided to go out of his way, even though I'd never met him and looked for some cybersecurity type jobs in my area that he thought maybe I'd be good for and helped me look at my resume and stuff like this. And that helped me land a vendor role for Microsoft, where I kind of started my path and career towards cybersecurity specific stuff.Justin Carroll:I had basically given up at that point on ever working in cybersecurity and had kind of thought that it just wasn't meant for me. So that was kind of a big break and a guy almost closed the application to apply for the job and then figured what's the worst they can say is no, that is kind of how I finally got to Microsoft and cybersecurity, where I was able to work as a vendor for the team evaluating kind of telemetry. And I was kind of given an opportunity to learn a lot and that eventually transitioned into when a position became available, where I started working full-time as a Microsoft employee and went from there.Natalia Godyla:So what in your soul search brought you to cyber security? Was it your background, the fact that you already had those foundations as a network admin, or was there something in particular in the cybersecurity world that just attracted you?Justin Carroll:I'd always found it fascinating. When I started university, they just launched the cybersecurity program. The quarter that I started there, and one of my friends who was a computer science major, basically called me up immediately and was like, "Hey, they just launched this. You need to do this." And there's the very popular culture aspect of it where everybody thinks it's fascinating and you sure there was a little bit of a grab with that. But I like learning how computers work and I like kind of the constant problem solving nature of everything. And the first class I took on it I was hooked and still remains that day where it's just, it's fascinating and it's really fun to just kind of continually work to see what attackers are doing. But I also, there's a huge aspect of it like I like helping people. I think it's important and having a role where I'm able to help millions or even potentially billions of people through better detections or stopping malware. It feels pretty great.Nic Fillingham:What other aspects Justin, of your path to security, your path to Microsoft, do you feel you're sort of bringing forward? I want to ask about you very briefly mentioned something about Halo 2 and I want to know what that was. And then I wonder if there were other sort of dare I say, sort of maybe unorthodox or non-traditional things that you worked on where you learned a bunch of bunch of tools or tricks of the trade that you're bringing forward to your work right now. Justin Carroll:So Halo 2 was a fun one. Back in those days, there were lots of what were called modders, who would mod their Xbox's to gain an unfair advantage. So I would use my networking know-how basically, and learned a lot of it too, when encountering a modder to kick them out of the game. I think it was possibly a little frowned upon, but I was tired of having cheaters constantly win, so I did a lot of research and I didn't know a whole lot about networking at that point, but I tried to not use it as a competitive advantage, but more to just level the playing field, but it was a great way to learn how firewalls worked and network traffic and building more on my understanding of computers. Justin Carroll:And then, kind of, that set a foundation for me, of understanding, there's always going to be stuff that I don't know and what I have done, but I did it all through college and continued all the way till basically getting full-time employment at Microsoft was I set up a lab environment and I would set up servers and clients and I would attack them and monitor the logs on my own little private lab on my machine and see what worked, what didn't, try and figure out why it worked, what didn't and try and build different tools to see how I could make it more effective or deal with different issues.Justin Carroll:Just kind of both playing attacker and defender at the same time on my network, all by myself, essentially and kind of learning from all of that data was massively important and anybody who's looking to get into security, I highly recommend both learning how to attack, on a safe, your own little lab environment where you're not hurting anybody. And what's it like to try and defend and find those attacks because both sides are-Nic Fillingham:Red Justin versus blue Justin. Justin Carroll:Exactly. Yes.Natalia Godyla:You noted earlier that just the sheer amount of data can be overwhelming, especially as you moved through your career and then came to Microsoft where we have billions of signals. So the same transition happens from Halo to now just the sheer scale and scope of your role and the amount of good that you can do. So, how did you handle that overwhelming amount of information, amount of impact that you can have?Justin Carroll:So when I was first brought on one of the things that made a significant difference was I had somebody that kind of instructed me in a lot of the ways of kind of how to work with the data, but I was also given quite a bit of an area for trial and error. So there was lots of opportunity to fail and to learn from what didn't work and to kind of keep building on that. And then any time that I got stuck or I would kind of just do everything I could to attempt to solve the problem or work with the data. If I kind of hit a wall that I couldn't climb on my own, I could go to him and then we would solve it together. So it was kind of both a mentoring and a guidance thing, but also kind of given that ability to experiment and try and learn. So that was kind of one of the biggest ways of learning to pivot on that data and understand it and consume it.Justin Carroll:And then honestly, collaboration with other folks on my team and other team was massively instrumental to be able to kind of learn what they had already learned or pass on my knowledge to them. And just that constant sharing and understanding because there is so much data, it's quite impossible almost to be an expert at all of it. So having those folks that you can reach out to you that are experts in each basically set of their data. So you can understand what the data is trying to tell you, because that's one of the things that is particularly difficult is to take the data and actually glean understanding from it. The data is trying to tell you something, you just need to make sure you're interpreting the message correctly.Natalia Godyla:How do AI and ML factor into your role into helping you manage this data and collaborating with other teams.Justin Carroll:So I work quite a bit with a lot of different data science folks on a few different teams to either use a lot of the models that they're creating to kind of a source, a lot of the malicious information or a particular attackers or stuff like that. And then also collaborating back in sharing my knowledge and intelligence to them to say, this is what an attack looks like. This is what it should look like in the data and kind of giving them the ideas and signals for what they should be looking in their data to kind of train those models. Justin Carroll:It's really important to have that partnership between security and data science for AI and ML to kind of help them understand the security sphere of it. And then they can kind of take the real math and data prowess that they've got and turn our knowledge into ML or AI to detect and surface a lot of these things. Nic Fillingham:If it's possible, Justin, how would you sort of summarize your guidance to other Justin Carroll's that are out there that are... They want to get into security, they're fascinated by cybersecurity in sort of a macro sense, but they feel either don't have a degree or they're not even sure what they should go study or they're trying to work at, how can they translate their current sort of career experience and sort of skills? Can you summarize that into some guidance of what folks should do to try and break in?Justin Carroll:Sure. One, if you're in school, remember that school is not going to teach you a lot of the stuff that you need to know. It's lots of taking what you're learning and building upon it outside. So if it's cybersecurity, that's an interest, try and experiment and fail. Cyber security is huge. There are so different facets of it. Find out the thing that kind of scratches the itch and piques your interest. For me, that was setting up a lab, right? Where I could play both the attacker, the defender, the person monitoring logs, the person setting up all the configurations to try and stop the attacks and was able to kind of see all different aspects of the industry. Nic Fillingham:So just jumping in, was that literally just a bunch of VMs on your machine or did you have multiple PCs sort of networked together? Just very quickly, what did that look like? How accessible is setting up a lab? I guess I'm what I'm asking. Justin Carroll:It is pretty accessible. So while I was in college, it was actually multiple machines and I had four different machines and I set up a router that you can pick up for 50 bucks and a smart switch that I could mirror the traffic on to understand everything for 100 bucks. So there's a little bit of cost. That was kind of my college setup. And as I was kind of learning where I at that point, it made a little more sense to do it with actual machines and for extra clarity. My college was only a couple of years ago. I did not go to college young. So the next route that I did once I headlined did my vendor role and was kind of like security is for me and I want to keep building on it.Justin Carroll:I did it all with VMs. So I just had a desktop computer that had okay specifications and I configured two clients, the domain controller, server on the device and then a mail server. And then basically you just connect to each client and then network them all together. So at that point you can use VirtualBox, you can use lots of different stuff. So the availability of doing that, it's actually pretty good. There isn't a lot of overhead costs or anything like that. You just have to have a okay computer.Natalia Godyla:What about resources to learn how to do all of that? Are there organizations or sites that someone could turn to, if they're interested in starting to do some of this starting to experiment with what they're interested in?Justin Carroll:Honestly, I would say one of the best resources that I had throughout was YouTube. It was a great place to get walkthroughs for every different thing. So like I wanted to learn how to set up a VM and configure it with networking to another VM. I turned to YouTube. I wanted to learn how to attack the VM using Kali Linux, YouTube. And there's a whole bunch of different channels out there that specifically focus on that. And then the other thing is because it's so much more open for creators to share content. You can find people who are at a similar level or maybe just a few steps ahead of you. So you can really kind of join along with other people. Justin Carroll:There are a few websites for coding, I think one's called hacking the box as far as attacking different things. And that was also kind of fun where a lot of the devices that need to be attacked we're already pre-configured for you. But for me, honestly, a lot of the fun was setting up those devices and then learning what I did that worked and didn't and what allowed it to be attacked and what I could do to stop that.Natalia Godyla:Quick plug Microsoft security also has a YouTube channel in case somebody would like to get any, how to content on our products.Nic Fillingham:Natalia may or may not have been involved in that channel, just full disclosure there.Natalia Godyla:Yeah. I couldn't help myself. But it is also great to hear that you found people to work with in the community as well. That's something that's been noted by a few of our guests, like Michelle Lamb, that as she was entering the space, she found mentors. She found conversations, people readily available to either work on a problem alongside her, or just answer questions. So I'm glad that you've also been able to turn to the community for that. So what's next for you? Is there a new challenge that you'd like to solve?Justin Carroll:Definitely want to work on the toolkit that I'm building and kind of continue that growth. It's been interesting to kind of see the hurdles I run into. And even last week I ran into one that felt insurmountable and was able to chat with one of the devs and solve in a few minutes and learned a whole lot and going forward, now I have that in my pocket. And then both-Nic Fillingham:Hang on. Did you say you went from found a new challenge, thought all this is insurmountable and then a few minutes later you solved it?Justin Carroll:With a little support from people that knew how to solve the problems. So collaborating with like one of the other devs on the team and basically having him kind of explain the part it felt like a giant wall, but really once you kind of have somebody to break it down a little bit for you, it was just like, "Oh, okay. I see what I'm missing here." And then it was just like, "Got it. Okay. Moving forward."Nic Fillingham:Oh, I see. So that that's more an endorsement. Yeah, I got it. Justin Carroll:Yeah. Yeah. It's more an endorsement of others teaching abilities and just kind of those times of being able to reach out to others for when you really get stuck and how much of a difference it can make. I had spent an hour on something and was just like, this is ridiculous. This should work. Why isn't it working? What's wrong with me. I'm not smart. And then just chatting with them a little bit and then figuring it out and then like, "Oh, okay. Oh, okay. That's actually pretty simple." I wasn't thinking about it in the right way and kind of getting that other perspective. Justin Carroll:And then what's next kind of going forward is a kind of continued partnership with a lot of the data science folks to, I think we've only scratched the surface in many ways as an industry on how data science and cybersecurity can work together. So I am very excited to kind of see what kind of stuff we can accomplish, whether it's, you know, surfacing attacks shortly after they happen, very early in the kill chain or understanding related behaviors and trying to understand who the might be, or I think most of all, the intent of the attack or adversary.Justin Carroll:Intent can sometimes be a very difficult to suss out, even for SOCs and their entire center. They have all these folks that are trying to figure out what happened. Why did it happen? What does it actually mean? So if we can have data science that can provide a lot of context on that, through understanding existing attacks and modeling what future ones might look like, I think there's some pretty exciting opportunities there.Nic Fillingham:All right, I'm doing it. We're coming to Teenage Mutant Ninja Turtles. You're a fan. How much of a fan are you, Justin?Justin Carroll:I'd say quite a fan. I do have a couple of figurines and a mint package unopened from '87 I think, something like that. And then have a Ninja Turtles tattoo on my back of Raphael. So that was kind of one of those moments where I was trying to think about what steps I wanted to take forward in life and things like that. And I had kind of thought about what are the things that actually make me happy? Justin Carroll:This was probably my mid 20s quarter life crisis kind of thing. And I was like, "I always liked the Ninja Turtles as a kid." They always brought me great joy. I still get excited about watching them. The movies are definitely a guilty pleasure. I realized they're not great. But now I'm talking about the original movies, not the new ones. We won't talk about the new movies. And it was just one of those like, "Yeah, I identify with this. This is a huge part of my life. It's been around since I was... it was started the year I was born." So I was just like, "All right, let's do it." And haven't regretted it at all.Nic Fillingham:I was going to ask who your favorite turtle was, but you've obviously... If you've inked Rafaelle on your back so that question is moot. I'm a Donatello guy. I've always been a Donatello guy.Justin Carroll:I would think of myself as Raf, but really I'm more of a Donatello. Ralph was kind of the cool guy with a little bit of an attitude, but really I was Donatello. When I was 10 dressed up for Halloween, I was Donatello. I'm definitely Donatello with a little bits Raf thrown in for good measure.Nic Fillingham:Well, this has been a blast. Thank you, Justin, for walking us down, Teenage Mutant Ninja Turtle memory lane, and Halo 2 memory lane and sharing your story with us. It was great. Wonderful to get your perspective. Great to have you as a part of the threat hunter team here at Microsoft and contributing in all the ways that you do. Thanks for joining us. I'm sure we'll talk to you again at some point on the Security Unlocked podcast, but keep doing you Cowabunga, dude.Justin Carroll:Thanks very much for having me. I appreciate it. It was great to talk to you all.Natalia Godyla:Well, we had a great time unlocking insights into security from research to artificial intelligence. Keep an eye out for our next episode.Nic Fillingham:And don't forget to tweet us @msftsecurity or email us at with topics you'd like to hear on a future episode. Until then stay safe.Natalia Godyla:Stay secure. See for privacy and opt-out information.
The last thing we all need this year is an identity crisis. Fear not, hosts Nic Fillingham and Natalia Godyla are here with Maria Puertas Calvo, Data Science Lead of Microsoft’s Identity Security and Protection Team, to learn how AI is being used to protect our personal identities. Maria also reveals previously undisclosed information – her favorite food and her famous top-secret recipe, so get ready to take notes! Later, the hosts bring back a previous guest, Geoff McDonald, ML Research Lead at Microsoft to unpack his career in cybersecurity and how game hacking led him to where he is now. In This Episode, You Will Learn: • How offline detections are used for account compromise prevention • The importance of multi-factor authentication • How Microsoft is taking a new approach with AI to identify threats with real-time prevention  • The problem with adversaries and malware attackers Some Questions We Ask: • How is Microsoft applying AI to solve problems for account compromise prevention? • How do humans play a role in labeling data sets? • How is Microsoft measuring success of their new enhanced AI? • What is the future for neural networks? Resources Maria’s Blog  Microsoft Security Blog  Nic’s LinkedIn  Natalia’s LinkedIn  Maria’s LinkedIn  Geoff’s LinkedIn Transcript(Full transcript can be found at and welcome to Security Unlocked. A new podcast from Microsoft where we unlock insights from the latest in news and research from across Microsoft Security engineering and operations teams. I'm Nick Fillingham.Natalia:And I'm Natalia Godyla. In each episode, we'll discuss the latest stories from Microsoft Security, deep dive into the newest threat Intel research and data science.Nic:And profile some of the fascinating people working on artificial intelligence in Microsoft Security. If you enjoy the podcast, have a request for a topic you'd like covered or have some feedback on how we can make the podcast better-Natalia:Please contact us at or via Microsoft Security on Twitter. We'd love to hear from you.Nic:Hello, Natalia. Welcome to episode eight of Security Unlocked. How are you?Natalia:I'm doing great. We're right about at Christmas. I am feeling it in my onesy right now.Nic:You're feeling Christmas in your onesy? Is it a Christmas onesy?Natalia:No. I feel like onesys just highlight the Christmas spirit. I mean, you're in PJs all weekend.Nic:We've been in work from home for seven years now. We're all in perpetual onesy land.Natalia:Well, I mean, I try to put in effort. I don't know about you.Nic:I don't put any effort. I wonder if we should issue a subscriber challenge. I wonder if we could hit 1000 subscribers. We might make a security unlocked onesy. I wonder what other swag we could do? What would be good for a security unlocked podcast?Natalia:All right. I mean, I guess I'm a little biased but the security blanket is clever. The ones that Microsoft gives away.Nic:I don't think I have one of those.Natalia:It's a blanket with security images on it.Nic:Images of security in it? Just images of very strong passwords. Images of two factor authentication. What about a horse blanket? Like a blanket you put over your horse?Natalia:What does that have to do with security?Nic:Under the saddle. I'm just following the blanket thread, that's all. I'm just thinking different types of blankets. In two episodes have already talked about the bratty pigs. I wonder if we could turn the bratty pigs into our mascot and on the security blanket there could be like an animated picture of the bratty pigs running away with a padlock and key or something.Natalia:Have I not, and excuse the pun, unlocked the new technology in blankets and animated pictures? Is that possible on blankets now?Nic:Did I say animated? I meant illustrated, I'm sorry. Oh wow, I bet you there's some brand new piece of printing technology that's over in like Japan or South Korea that we haven't got over here yet where they've got animation on their blankets, that would be good. What about one of those automatic cat feeders for when you go away on holiday and it dumps a little bit of dry food into their bowl every 12 hours? And then we just put Security Unlocked on the side of it.Natalia:As long as it has our logo on, it fits.Nic:You know what? Also, this is our last episode for 2020.Natalia:How'd you feel about it?Nic:About this episode or about the year of 2020?Natalia:Well, the year 2020 is probably too much to unpack. What about our podcast adventure in 2020?Nic:Yeah, I've enjoyed it greatly. I listened to the first couple of episodes just the other day. And while they were great, I certainly heard an evolution in just eight episodes from that humble first back in October. So yeah, I've definitely enjoyed the trip. I'm very much looking forward to 2021. What about you?Natalia:I feel like our guests are making me smarter. With each new episode. I've got a few more terms under the belt. Terms I'd heard before but never got that clarity from experts and what the definition is especially as they're moving around. We see that with a lot of the machine learning and AI terms. Like neural networks when we're talking to experts, they have different lenses on what that should mean.Nic:The other thing that I found fascinating is everyone that you and I have reached out to internally, Natalia, and said, "Hey, do you want to be a part of this podcast?" Everyone said, Yes. Everyone has said, "Yeah, I'd love to share my story of how I got into security. I'd love to share my story of how I got to Microsoft." I love that we've spoken to such a incredible variety of people that have come to security and to Microsoft from just... I mean, everyone has a completely different story and everyone's been so willing to tell it. So I'm just very, very happy that we've been able to meet these great people and have these conversations.Natalia:Yes. And even in their diversity, I've been happy to see that there are really positive themes across the folks that wants to be in security that are in the security space. They're all so passionate about what they do and really believe in the mission, which is just great to see. And like you said, there's just awesome community. The fact that they want to go out and have these conversations and are always open to receiving questions from you guys. So please keep them coming. Our experts are equally as hungry as we are to hear not just feedback but questions on the topics that we discuss.Nic:So on today's episode, we chat with Maria Puertas Calvo. Fantastic conversation, very excited to have Maria on the podcast. I'm not sure if many folks picked up but a lot of the experts we've spoken to so far have been more on the endpoint detection side of the house. We've talked to folks over in the defender team and those who sort of look at the email pipeline. Maria and her team focused on identities, so protecting identities and protecting our identity platforms. And so she's going to talk about how AI and ML are used to protect identity. And then after Maria, we talked to...Natalia:Jeff McDonald. So he is a member of the Microsoft defender for endpoint research team. And he's joined us on a previous episode to talk about unmasking malicious threats with MC and ML. And today, he's chatting with us about his career in cybersecurity, which started with game hacking. So making changes in the game to get more skills, get new characters and he's got some amusing stories as to how far he took that. But it's also a theme we're seeing across a few of our guests that game hacking seems to be a gateway to cyber security.Nic:Yeah, hopefully the statute of limitations on game hacking has well and truly expired on the various games that Jeff mentions in his interviews. I hope we're not getting him in trouble. Enjoy the pod, and we'll see you all in 2021.Nic:Maria Puertas Calvo, thank you so much for joining us. Welcome to the Security Unlocked podcast.Maria Puertas Calvo:Hi, thank you for having me.Nic:If you could tell us about your role at Microsoft and what your day to day looks like in the team you're in. The mission and sort of scope of that work, that'd be great.Maria Puertas Calvo:Yeah, absolutely. So I am a principal data science manager in identity security and protection. So I lead a team of five data scientists that work within a big engineering team. And our big mission is to protect all of Microsoft's users from account compromise and other things like the abuse and fraud. As a data science team, we just analyze and look through all the huge amount of data that we get from all our customer logs and everything. And then we use that to build automated statistical based models or machine learning models or heuristic made models that are trying to detect those bad actions in our ecosystem. So compromised attacks or malicious bots that are trying to do bad things in our identity systems.Natalia:And Maria, we understand that your team also recently authored a blog on enhanced AI for account compromise prevention. So can you talk a little bit about what that blog entails, how we're applying AI to start solving some of these problems?Maria Puertas Calvo:Yeah, we're actually really excited about this work. But it just went into production recently and it has really enhanced what we call the bread and butter of really what we do. Which is trying to prevent compromise from happening in the ecosystem. Basically, we have been using artificial intelligence and AI to build detections for a pretty long time. And everything that we do, we try to start with whatever the long hanging fruit. We do offline detections, which are basically using the data after authentications or attacks already occurred and then detect those bad attacks and then we will inform the customer or make the customer reset their password or do some type of remediation.Maria Puertas Calvo:But being able to put AI at the time of authentication and so meeting that end goal that we're trying to not just detect when a user has been compromised and remediate it but we're actually able to prevent the compromise from happening in the first place. So this new blog talks about this new system that we've built. We already had real time compromised detection but it wasn't using the same level of artificial intelligence.Natalia:So is it correct to say then that in the past we had been doing is identifying a known attack, a known threat, and then producing detections based on that information and now we're trying to preempt it? So with this even more intelligent AI, we're trying to identify the threat as it's happening, is that correct?Maria Puertas Calvo:Yeah, that's correct. So we did already have real time prevention but most of our artificial intelligence focus used to be in the, after the fact. Now we have been able to move this artificial intelligence focus also to the real time prevention. And what we have achieved with this has really improved the accuracy and the precision of this detection itself. Which means now we're able to say that the signings that we say are risky, they're way more likely to actually be bad than before. Before we would have more noise and more false positives and then we would also have some other bad activities that would go undetected.Maria Puertas Calvo:With this new artificial intelligence system, we have really increased the precision. Which means, now if a customer says, "Oh, I want to block every single medium risk login that comes my way that is trying to access my tenant." Now, fewer of their real users are going to get blocked and more actual attackers are going to get blocked. So we've really improved the system by using this new AI.Natalia:What's changed that's increasing the precision?Maria Puertas Calvo:Yeah, so we actually published another blog with the previous system which was mostly using a set of rules based on user behavior analytics. So the main detection before was just using a few features of the signing itself and comparing them to the user history. So if you're coming from a new IP address, if you coming from a new location, if you're coming from a new device, there was like a deterministic formula. We were just using a formula to calculate a score which was the probability of how unfamiliar that finding was. Now we're taking way more inputs into account. So we're using... It depends on which protocol you're using.Maria Puertas Calvo:It has more intelligence about the network, it has some intelligence about what's going on. for example, if you're coming from an IP address that has a lot of other traffic that AD is seeing, it has also information about what AD is saying from that IP address. Does it have a lot of failed logins or is it doing something weird? And then instead of us manually setting a mathematical formula or rules in order to build that detection, what we do is we train an algorithm with what is called label data. So label data is just a set of authentications, some are good and some are bad and they're labeled as such. So we use that label data to tell the algorithm, "Hey, use this to learn," Right? That's how machine learning works.Maria Puertas Calvo:So the algorithm trains and then it's able to use that data to decide in real time if the authentication is good or bad.Nic:Yeah, thank you. And then where, if any, do human analysts or humans in specialty roles, if it's data science or analytics, when do they come in to either verify the results or help with labeling new sets of data? So you've got your known goods, you've got your known bads and I assume you end up with a bunch of unknowns or difficult to classify one way or the other. Is that a role for a human analyst or human data scientists to come in and create those new labels?Maria Puertas Calvo:Yeah, even though getting all this labels is extremely important. That is not really what... The data scientist is not there just classifying things as this is good, this is bad, just to get labels to feed it to the algorithm, right? What the data scientist does that is very crucial is to build the features and then train this machine learning model. So that is the part that is actually really important. And I always really try to have everybody in my team to really understand and become a great domain expert on two things, One is the data that they have to work with. It is not enough to just get the logs as they come from the system, attach the label to it and then feed it to some out of the box classifier to get your results.Maria Puertas Calvo:That is not going to work really well because those logs by themselves don't really have a lot of meaning. If the data scientist is able to really understand what each of the data points that are in our laws, sometimes those values, they're not coded in there to be features for machine learning. They're just added there by engineers to do things like debugging or showing locks to the user. So the role of the data scientist is really to convey those data points into features that are meaningful for the algorithm to learn to distinguish between the attack or the good. And that is the second thing that the data scientist needs to be really good at. The data scientist needs to have a very good intuition of what is good and how that looks in the logs versus what is bad and how the looks in the logs.Maria Puertas Calvo:With that knowledge basically knowledge of what the data in the logs mean and the knowledge of what attack versus good look in that data, then that is the feature engineering role. You transform those logs into all their data points that are calculations from those logs that are just going to have a meaning for the algorithm to learn if something is good or an attack. So I can give an example of this, it's very abstract. For example, when I see an authentication in Azure AD logs maybe one of the columns that I'd want him to know is like IP address, right? Every single communication over the internet comes from some client IP address which will be the IP address that's assigned to the device that you are on at the time that you're doing an authentication.Maria Puertas Calvo:There are billions, if not trillions of IP addresses out there. And each one is just some kind of number that is assigned to you or to your device and it doesn't really have any meaning on its own. It's just like if you have a phone number, is that a good or a bad phone number? I don't know, that's just not going to help me. But if I can actually go and say, "Okay, this is an IP address but is this an IP address that Nick use yesterday or two days ago? How often have I seen Nick in this IP address? What was the last time I saw Nick in this IP address?" If you can just play with those logs to transform it into this more meaningful data, it's really going to help the model understand and make those decisions, right?Maria Puertas Calvo:And then you also end up with fewer things to make decisions on, right? Because if I just had that one IP address to train the model, maybe my model will become really good at understanding which IP addresses are good and bad but only among the ones that we have used to train that model. But then when a new one comes in, the model doesn't know anything about that IP address, right? But if we instead change that into saying, "Okay, this is a known IP address versus an unknown IP address," And then now, instead of having trillions of IP addresses, we just have a value that says, Is it known or unknown. Then for every single new log in that comes in, we're going to be able to know if it's known or unknown.Maria Puertas Calvo:We don't really need to have seen that IP address before, we just need to compare it to the user history and then make that determination of it is this known or unknown and that ends up being much more valuable for the model.Natalia:So just mapping out the journey you've talked about. So we've gone from heuristics signature based detections to user analytics and now we're in a space where we're actively using AI but continuously optimizing what we're delivering to our customers. So what's next after this new release of enhanced AI? What is your team working on?Maria Puertas Calvo:So lots of things but one thing that I am really interested in that we're working on is making sure that we're leveraging all the intelligence that Microsoft has. So for example, we built a system to evaluate in real time, the likelihood that a finding is coming from an attacker. But all of that is just using the data that identity processes like Azure Active Directory sign ins and what's happening the Azure Active Directory infrastructure. But there's so much more that we can leverage from what is happening across the ecosystem, right? Like the user who signs into Azure Active Directory is probably also coming in from a Windows machine that probably has Microsoft dependent Defender ATP installed on it. That it's also collecting signal and it's understanding what it's happening to the endpoint.Maria Puertas Calvo:And at the same time, when the sign in happens then the sign in doesn't happen just to go to Azure AD, right? Azure AD is just the door of entry to everything, Usher, Office, you name it. Third party applications that are protected by things like Microsoft Cloud App Security. And all of the security features that exist across Microsoft are building detections and collecting data and really understanding in that realm, what are the security threats and what's happening to that user? So there is a journey, right? Of that sign in. It's not just what's happening in Azure AD but it's everything that's happening in the device. What's happening in the cloud and in the applications that are being accessed after.Maria Puertas Calvo:So we're really trying to make sure that we are leveraging all that intelligence to enhance everything that we detect, right? And that way, the Microsoft customer will really benefit from being a part of the big ecosystem and having that increased intelligence should really improve the quality of our risk assessment and our compromise detections.Nic:Maria, how much of this work that you talked about in the blog and the work that your team does is trying to mitigate the fact that some folks still don't have multi factor authentication? Is any of this a substitute for that?Maria Puertas Calvo:We know from our own data studies that accounts that are protected by multi factor authentication, which means every time they log in, they need to have a second factor, those accounts are 99.9% less likely to end up compromised because even if their password falls in the hands of a bad actor or get gassed or they get phished, that second factor is going to protect them and it's way more likely to stop the attack right there. So definitely, this is not supposed to be a substitute of multi factor authentication. Also, because of that, our alerts do not... They still will flag a user if the sign in was protected by multi factor authentication but the password was correct. Because even if there's multi factor authentication, we want to make sure that the user or the admin know that the password was compromised so they're able to reset it.Maria Puertas Calvo:But the multi factor authentication is the tool that is going to prevent that attack. And you asked earlier about what's next in other feature things and one thing that we're also really working on is, how do we move past just detecting these compromises with the password of using multi factor authentication as a mitigation of this risk, right? Like the way a lot of the systems are implemented today is if you log in and we think your log in is bad but then you do MFA. That is kind of like a reassuring things that we committed a mistake, that was a false positive and that's a remediation event. But the more people move to more MFA and more password less, our team is starting to think more and more of what's the next step?Maria Puertas Calvo:How are attackers are going to move to attacking that multi factor authentication. It is true that multi factor authentication protects users 99.9% of the time today but as more people adopt it, attackers are going to try to now move to get to bypass our multi factor authentication. So there's many ways but the most popular multi factor or second factor that people have in their accounts is telephone based. So there's SMS or there's a phone call in which you just approve the Sign In. There are phishing pages out there that are now doing what is called real time men in the middle attack in which you put your username and password, the attacker grabs it, puts it in the actual Azure AD site and then now you're being asked to put your SMS code in the screen. So the attacker has that same experience in their phishing site, you put in your code and the attacker grabs the code and puts it in Azure AD sign in page and now the attacker has sign in with your second factor, right?Maria Puertas Calvo:So two challenges that we're trying to tackle is, one, how do we detect that this is happening? How do we understand that when a user uses their second factor, that is not a mitigation of the risk? It's more and more possible with time that attackers are actually also stealing this second credential and using it, right? So we need to make more efforts in building those detections. And the second really big thing is, what then, right? Because if we actually that the attacker is doing that, then what is the third thing that we asked you? Now you've given us a password, you've given us a second factor, if we actually think that this is bad, but it is not. What is the way for the user to prove that it's them, right?Maria Puertas Calvo:So we need to move and I think this is extremely interesting, we need to move to from a world in which the password is the weak crab and everything else is just considered good. which today, it's very true. If you have a second factor, that is most likely going to be just fine but in the future, we we need to adapt to future attacks in which this won't be the case. So we need to understand what is the order of security of the different credentials and what is the remediation story for attacks that are happening with these second factors.Nic:I'd like to propose that third challenge, that third factor, should be a photograph of you holding today's newspaper doing the floss or some other sort of dance craze that's currently sweeping the nation.Maria Puertas Calvo:Sure, we'll add it to the bar code.Nic:I think that would just stamp out all identity theft and fraud. I think I've solved it.Maria Puertas Calvo:You did. I think so.Natalia:I think you'll be bringing back newspapers along with it.Nic:Yes. Step one is to reinvigorate the print newspaper industry. That's the first step of my plan but we'll get there.Natalia:So Maria, in your endeavors? How are you measuring success, for instance, of the new enhanced AI that your team has developed?Maria Puertas Calvo:Yeah, so our team is extremely data driven and metric driven and everything we do, we're trying to improve on one metric, right? The overall team mission really is to reduce the amount of users who fall victims of compromised account or what we call unauthorized access. So we have a metric that we all review every single day, we have a huge dashboard that is everybody's homepage in which we see in the last three months, what percentage of our monthly active users fell victim to compromised account and our main goal is to drive that metric down. But that is really the goal of the whole team including the people who are trying to make users adopt MFA and conditional access and other types of security measures.Maria Puertas Calvo:When we look into detection metrics and the ones like the AI detection metrics, we mostly play with those precision and recall metrics that are also explained in the blog. So precision is the percentage of all of the detected users or detected signings that you detected as bad that are actually bad, right? Out of everything that, let's say, you would block, how many of those were actually bad? So it really also tells you how much damage you're doing to your good customers. And the other one is recall and recall is out of all the bad activities that are out there, so let's say all the bad sign ins that happen in a day, how many of those that your system catch?Maria Puertas Calvo:So it's a measure of how good you are at detecting those bad guys. And the goal is to always drive those two numbers up. You want to be really high precision and you want to be really high recall. So every time we'll have a new system and a new detection or whatever it is or we perform improvements in one of our detection, those are the two metrics that we use to compare the old and the new and see how much we've improve.Natalia:And how are we getting feedback on some of those measures? And what I mean by that is the first one you mentioned. So precision, when you're saying how many were actually bad and we need to figure out how many were the true positive? How do we know that? Are we getting customer feedback on that or is there a mechanism within the product that lets you know that it was truly a bad thing that was caught?Maria Puertas Calvo:Yeah, so the same label and mechanisms that I was talking about earlier that we need both labels to be able to train or supervise machine learning models, we also need those labels in order to be able to evaluate the performance of those machine learning models. So knowing at least for a set of our data, how much is good and how much is bad and understanding what our systems are doing to detect the good and the bad. So one of the mechanisms is, as I was talking, the manual labeling that we have in place but the other one you mentioned is customer feedback, absolutely. Actually, one of the first thing we did when we launched editor protection is to include feedback buttons in the product.Maria Puertas Calvo:All of our detections actually go to an Azure Portal UX in the identity protection product and admins there can see all of the risky sign ins and all of the risky users and why they were detected as risky. Everything that my team is building gets to the customer through that product. And that's where the admin can click buttons like confirm safe or confirm compromised. Those are labels that are coming back to us. And users now also, there's a new feature in entity protection called My Finance. And users can go to my sign ins and look at all their recent signings that they did and they can flag the ones that they think it wasn't them. So if they were compromised, they can tell us themselves, this was not me.Maria Puertas Calvo:So that is another avenue for us to understand the quality of our detections. And then we're extremely customer obsessed as well. So even, it's not just the PMs in our team who have customer calls. The data scientists, many, many times get on calls with customers because the customers really want to understand what's the science behind all of these detections and they want to understand how it works. And the data science teams also wants the feedback and really understand what the customer thinks about the detection. If we're having false positives, why is that? It's really challenging too in the enterprise world because every tenant may have a different type of user base or different type of architecture, right?Maria Puertas Calvo:We had a time that we were tracking... We always track what are the top 10 tenants that get flagged by the detections. For example, airlines used to be a big problem for us because they had so much travel that we had a lot of false positives, right? We were flagging a lot of these people who because they're flying all over the world and signing in from all over the world. So it would trigger a lot of detections but there are other customers in which this is not the case at all. All of their users stay put and they're just only logging in from the corporate network because it's a very protected environment. So this quality of detections and this precision and recall can really vary customer by customer.Maria Puertas Calvo:So that is another challenge that I think we need to focus more in the future. How do we tune our detections in order to make more granular depending on what the industry is or what type of setup the customer or the tenant has.Nic:Changing subjects just a little bit and maybe this is the last question, Maria. I noticed on your Twitter profile, you refer to yourself as a guacamole eater. I wondered if you could expand upon that. There are very few words in your bio but there's a lot of thought gone into those last two words. Tell us about eating guacamole.Maria Puertas Calvo:Well, what can I say? I just really love guacamole. I think I may have added that about a year ago, I was pregnant with my twins who were born five months ago and when you're pregnant with twins they make you eat a lot of calories, about 3000 calories a day. So one of the foods that I was eating the most was guacamole because it's highly nutritious and it has a lot of calories. I went on a quest to finding the best recipe for guacamole and-Nic:Okay, walk us through your best guacamole recipe. What's in it?Maria Puertas Calvo:Absolutely. So the best guacamole recipe has obviously avocado and then it has a little bit of very finely chopped white onion, half jalapeno, cilantro and lime and salt. That's it.Nic:No tomatoes?Maria Puertas Calvo:No tomatoes. The tomatoes only add water to the guacamole, they don't add any flavor.Nic:What about then a sun dried tomato? No liquid, just the flavor? Is that an acceptable compromise?Maria Puertas Calvo:Absolutely not. No tomatoes in guacamole. The best way to make it is, you first mash the jalapeno chili with the cilantro and the onion almost to make a paste and then you mix in the avocado and then you finally drizzle it with some lime and salt.Nic:Hang on. Did you say garlic or no garlic?Maria Puertas Calvo:No garlic, onion.Nic:No garlic, I see. So the onion is the substitute for I guess that's a savoriness? I don't know how you classify... What's garlic? Is it Umami? I don't know the flavor profile but no garlic? Wow, I'm making guacamole when I'm at my house.Natalia:Well, you heard it here first guys. Maria's famous guacamole recipe.Nic:I think we'll have to publish this on Twitter as a little Easter eggs for this episode. It'll be Maria's definitive guacamole recipe.Maria Puertas Calvo:Now the secret is out.Nic:Well, Maria, thank you so much for your time. This has been a fantastic chat I think. I have a feeling we're going to want to talk to you again on the podcast. I think we'd love to hear a bit more about your personal story and I think we'd also love to learn more about some of the AI techniques that you talked to us about but thank you so much for your time.Maria Puertas Calvo:Yeah, of course, this was a pleasure. I had a great time and I'll come back anytime you want me. Thank you.Natalia:And now let's meet an expert from the Microsoft Security Team to learn more about the diverse backgrounds and experiences of humans creating AI and tech at Microsoft. Today, we're joined by Jeff McDonald, who joined us on a previous episode, unmasking malicious scripts with machine learning to talk to us about anti-malware scan interface or AMC. Thank you for joining us again on the show, Jeff.Geoff McDonald:Yeah. Thank you very much. I really enjoyed being here last time and excited to be here again.Natalia:Great. Well, why don't we start by just giving a quick refresher to our audience? Can you share what your role and day to day function is at Microsoft?Geoff McDonald:I lead a team of machine learning researchers and we build our machine learning defenses for Microsoft Defender antivirus product. So we built lightweight machine learning models which go into the antivirus product itself which run on your device with low memory and lower CPU costs for inference. We also deploy a lot of machine learning models into our cloud protection platform where we have clusters of servers in each region around the world. So that when you're scanning a file or behavior on your device, it sends metadata about the encounter up to our cloud protection in real time to the closest cluster to you. And then we do real time running of all of our machine learning models in the cloud to come back with a decision about whether we should stop the behavior or attack on your device.Geoff McDonald:So we're a small team of probably about five of us. We're a mix of threat researchers and machine learning and data science experts. And we work together to design new protection scenarios in order to protect our customers using machine learning.Nic:Jeff, when you go to a security conference, some kind of industry get together, do you describe yourself as a machine learning engineer? What do you use when you're talking to other security professionals in your field? Is machine learning... Is it sort of an established subcategory or is it still sort of too nascent?Geoff McDonald:Yeah. I used to call myself maybe a threat researcher or a security researcher when I would present at conferences and when I would introduce myself. But I'd say nowadays, I'd be more comfortable introducing myself as a data scientist because that's my primary role now. Although I come from a very strong background in the security and security research aspect, I've really migrated to an area of work where really machine learning and data science is my primary tool.Natalia:What's driven that change? What prompted you to go deeper into data science as a security professional?Geoff McDonald:So when I first started at Microsoft, I was a security researcher. So I would do a reverse engineering of the malware itself. I would do heuristics, deep analysis of the attacks, and threat families and prepare defenses for them. So I think learning pretty early on while doing all the research in response to these attacks, it was very clear that the human analysis and defense against all these attacks was really not scalable to the scale that we needed. So it really had to be driven by automation and machine learning, in order to be able to provide a very significant protection level to our customers. So I think that really drove the natural solution where all these human resources, these manual analysis doesn't scale to where we need it to be and where we want our protection level to be.Geoff McDonald:So it really encouraged finding the automation and machine learning solution. And I have previously had some experience with machine learning. At the time, it was kind of a natural fit where I began a lot of exploration of the machine learning application to protect it against these threats and then pivoted into that as my primary role eventually, as it was quite successful.Natalia:So your unique set of skills, data science and security, is one that's definitely sought after in the security space. But considering the fact that we're still trying to fill just security jobs, it's definitely a challenge. So do you have any recommendations for companies that are looking for your set of skills and can't find a unicorn like yourself that has both? And if were looking for multiple people, how should these teams interact so that they're leveraging both skills to protect companies?Geoff McDonald:When we look to fill new positions on our team, we try to be really careful to try to be as inclusive as possible to a lot of different candidates. So when we're pushing our new data science positions where we're looking for the data science experience, like in the machine learning and data science application, you'll see in our job applications, we don't actually require cybersecurity experience for our job positions. We're really looking for someone who has a really great understanding of the data and good understanding of ML. And being able to have a strong coding background in order to be able to implement these pipelines and machine learning models and try out their experiments and ideas in ways that they can implement and take them end to end to deploying them.Geoff McDonald:So really, for people that were looking to join our team, often, you don't actually necessarily have to have a background in cybersecurity for all of our positions. Sometimes we're looking for really strong data scientists who can pick up the basics of security and apply it in a very effective way. But we would also want our team have different sets of people who are more experienced in the security background to help drive some of the product and feature and industry and security trends for the team as well. Our team currently has quite a mix of backgrounds where there's some threat researchers and there's some pure data scientists who have come from related fields who actually haven't come from a cybersecurity background specifically.Nic:I wonder if we can back it up. If we can go back in time and start with you, your story, how did you first get into security, get interested in security? Did it start in elementary school? Did it start in high school? Did it start in college? Did you go to college? Can we back up and learn about the young Jeff McDonald?Geoff McDonald:I grew up in a small town near Calgary, Alberta, Canada. I guess it started with my family being a software developing family, I would say. Like my dad had his own software company and as a result, we were really lucky to have the opportunity to learn to code from a young age. So, we would see our dad coding, we knew that our dad coded so we're really interested in what he was doing and we wanted to be able to learn and participate.Nic:When was that Jeff? We're talking in 80s, 90s?Geoff McDonald:So that would be when I was probably around 10 years old when I started coding. And that would be I guess, 96 or so.Nic:I'm trying to learn like was that on some cool, old Commodore 64 hardware or were we well and truly in the x86 era at that point?Geoff McDonald:Yeah. I mean, an x86 I do believe. So it's just Visual Basic which is very simple coding language. The classic Visual Basic 6.0, we're really lucky to be able to learn to code at a pretty young age, which is awesome. And although my brother went more into... My older brother was about two years older, a big influence on me coding wise as well. He was really into making, you might say, malware. We both had our own computers, we had often tried to break into each other's computers and do things. My brother created some very creative hacks, you can say. Like, one thing I remember is he burned a floppy disk, which would have an autorun on it and the way that I'd protect my computer is a password protected login.Geoff McDonald:But back in those days, I think it was windows 98 at the time, it really wasn't a secure way of locking your computer where you have to type in your password. You can actually insert a diskette and would run the autorun and you could just terminate the active process. So my brother created this diskette and program, which would automatically be able to bypass my security protocols and my computer, which I thought was pretty funny.Nic:Is he still doing that today? Is he still red teaming you?Geoff McDonald:No. Not red teaming me anywhere, luckily.Natalia:So what point were you like, "Well, all of these things that I've been doing actually apply to something I want to be doing for a career?"Geoff McDonald:Yeah. So although was in a really software development friendly household. My dad was really concerned about the future of software development. He was discouraging us from going into software development as a primary career path at the time. Going into university I was mostly considering between engineering and business. I ended up going into engineering because I really liked the mathematical aspect of my work and it is a mix of coding and math, which is kind of my two strong suites. So I went into electrical engineering program, during my electrical engineering for four years is when I really changed from doing game hacking as my hobby to doing software development for reverse engineering tools. So as my hobby, I would create a reverse engineering tools for others to use in order to reverse engineer applications. So I went to universities in Calgary, Alberta there. And in Alberta, the primary industry of the province is oil and-Nic:Is hockey.Geoff McDonald:Good one. Yeah. So in Alberta, the primary industry in the sector is really oil and gas. There's a lot of oil and gas, pretty much all engineers when they graduate, the vast majority go into the oil and gas industry. So really, that's what I was thinking of that I'd probably be going into after I graduate. But either way, I continued the reverse engineering tool development, I did some security product kind of reverse engineering ideas as well. Approaching graduation, I was trying to figure out what to do with my life. I loved control systems, I loved software development, I loved the mathematical aspects and I want to do grad school. So then I looked at programs in security because my hobby of reverse engineering security, I didn't really take very seriously as a career.Geoff McDonald:I didn't think it could be a career opportunity, especially being in Alberta, Canada where oil and gas is the primary sector, there's not much in the way of security industry work to be seen as far as I could tell at the time in the job postings and job boards. So I ended up going for a master's in control systems continuing electrical engineering work. So basically, it's more like signal processing work where you're doing analyzing signals doing fault detection, basically, mount vibration sensors to rotating machines was my research. And then from the vibration signal, you're trying to figure out if there's a fault inside the motor or the centrifuge or the turbine or whatever it's attached to.Geoff McDonald:And in that field, there was a lot of machine learning in the research area. So that's where I got my first exposure to machine learning and I loved machine learning but that wasn't my primary research focus for my topic. And then approaching graduation, I started looking at jobs and I happen to get really lucky at the time that I graduated because there happened to be a job posting from Symantec in Calgary. And when looking at the requirements for the job postings, it had all of the reverse engineering tools and assembly knowledge and basically everything I was doing as a hobby, had learned through game hacking and developing these reverse engineering tools. It was looking for experience in only debug assembly. I'm like, "Oh, my goodness. I have all those skills. I can't believe there's actually a job out there for me where I could do my hobby as a career." So I got really lucky with the timing of that job posting and so began my career in cybersecurity instead of oil and gas.Nic:So you talked about the adding sensors parts to, I guess, oil and gas related sort of instrumentation. And then there was some machine learning involved in there. Is that accurate? So can you expand upon that a little bit, I'd love to learn what that look like.Geoff McDonald:So basically, the safety of rotating machines is a big problem. There was an oil and gas facility actually in Alberta which has centrifuges which spins the... I'm sure I'm not using the right terminology, but it spins some liquid containing gas to try to separate the compounds from the water, I think. And they had one of these... Actually, the spindle of the centrifuge broke and then it caused an explosion in the building and some serious injuries. So it was really trying to improve the state of the art of the monitoring of the health of a machine from the mounted accelerometers to them.Geoff McDonald:Two of the major approaches were machine learning, where you basically create a whole bunch of handcrafted features based on many different techniques and approaches and then you apply a neural network or SVN or something like that to classify how likely it is that the machine is going to have a failure or things like that. Now, I think at the time the machine learning was applied but it wasn't huge in the industry yet because machine learning in application to signals, that was, especially in convolutions, not as mature as it is now. The area I was working on was de-convolutions. A lot of machine learning models involve doing... At least a lot of machine learning models nowadays would approach that problem as a convolutional neural network. The approaches that I was working on next one was called a de-convolution approaches.Geoff McDonald:So I was able to get a lot of very in depth research into convolutions and what the underlying mean. And that has helped a lot with the latest model architectures where a lot of the state of the art machine learning models are based on convolutions.Natalia:So what was that a convolutional neural network? Can you define what that is?Geoff McDonald:So convolution is basically where you're applying a filter across the signal. It could be an image or it could be a one dimensional signal. So in this case, it's a one dimensional signal where you have... Well, at least it's a one dimensional signal if you have a single accelerometer on a single axis for the machine. You think of it like the classic ECG where you have a heartbeat going up and down. It's kind of like that kind of signal you can imagine which is the acceleration signal. And then you basically learn to apply a filter to it in order to maximize something. What filter you apply can be learned in different ways. So in a convolutional neural network, you might be learning the weights of that filter, how that filter gets applied based on back propagation through whatever learning goal you're trying to solve.Geoff McDonald:In a typical CNN model, you might be learning something like 1000 of these filters where you're adjusting the weights of all these filters through back propagation according to... To try to minimize your loss function. I guess in my research area, I was working to maximize, design a filter through de-convolution to maximize the detection of periodic spikes in the vibration signal. Meaning that something like an impact is happening every cycle of the rotor, for example.Nic:Well, so convolution is a synonym for sort of complexity. So de-convolution, is that a oversimplification to say that it's about removing complexity and sort of filtering down into a simpler set, is that accurate?Geoff McDonald:I wouldn't say it's so similar to the English language version of it. It's a specific mathematical operator that we apply to a signal. So it's kind of like you're just filtering a signal. And de-convolution is sort of like de-filtering it. It's my best way to describe it.Nic:Oh, right. Okay, interesting. De-filtering it. Could you take a stab at just giving us your sort of simplest if possible definition of what a neural network is?Geoff McDonald:Okay. A simplest stab of a neural network, okay.Nic:And Jeff, there's very few people have asked that question of but you're one of them.Geoff McDonald:Okay, cool. When you look at the state of the art, you'll actually find that neural networks themselves are not widely used for a lot of the problems. So when it comes to like a neural network itself, the best way I might describe it is that it's basically taking a bunch of different inputs and it's trying to predict something. It could be trying to predict the future stock price of Tesla, for example, if they're trying to predict whether Tesla's going to go up or down or they could be trying to predict it. Especially in our Microsoft defender case, we're trying to predict, "Based on these features, is this malicious or not?" Is our type of application.Geoff McDonald:So it's going to mean taking a whole bunch of inputs like, "Hey, how old is this file in the world? how prevalent is this file in the world? What's its file size? And then what's the file name?" Well, maybe I'll say, "Who's the publisher of this file?" Well, it's going to take a whole bunch of inputs like that and try to create a reasoning... It's going to try to learn a reasoning from those inputs to whether it's malware or not as the final label. We do it through a technique called back propagation because we have imagined a million encounters where we have those input features. So then we use these known outputs and inputs in order to learn a decision logic to best learn how to translate those inputs to whether it's Malware or not.Geoff McDonald:So we do this through a lot of computers or sometimes GPUs as well in order to learn that relationship. And a neural network is able to learn nonlinear relationships and co-occurrences. So for example, it's able to learn a logic like is it more than 10,000 file size? And is the publisher not Microsoft? And the age is less than seven days, then we think it's 70% malicious. So it's able to learn sort of more complex logic like that, where it can create and conditions and create more complex logic depending on how many layers you have in that neural network.Natalia:Do you think there's a future for neural networks? It sounds like right now you see a specific set of use cases like image recognition but for other use cases it's been replaced. Do you think the cases you described right now like image recognition will eventually be replaced by other techniques other than neural networks?Geoff McDonald:I think they'll always play a role or derivatives of them will play a role. And it's not to say that we don't use neural networks at all. Like in our cloud protection platform, you'll find tons of logistic regression single neuron models, you'll find GBM models, you'll find random forest models. And we've got our first deep learning models deployed. Some of our feature sets have a lot of rich information to them and are really applicable to the CNN, the convolutional neural network model architecture and for those, we will have a neural network at the end of the month. So it still definitely plays its specialty role but it's not necessarily what's driving the bulk of protection. And I think you'll probably find the same for most machine learning application scenarios around the industry. That neural network is not key to most problems and that it's not necessarily the right tool for most problems but it does still play a role and it definitely will continue to play a role or derivatives of it.Nic:My brain's melting a bit.Natalia:I want to ask for a definition of almost every other term but I'm trying to hold back a bit.Nic:Yeah, I've been writing down like 50 words that Jeff has mentioned like, "Nope, I haven't heard that one before. Nope, that one's new." I think, Jeff, you've covered such a lot of fascinating stuff. I have a feeling that we may need to come back to you at other points in the future. If we sort of look ahead more in general to your role, your team, the techniques that you're sort of fascinated in? What's coming down the pike? What's in the future for you? Where are you excited? What are you focused on? What are you going to see in the next six, 12 18, 24 months?Geoff McDonald:One of the big problems that we have right now is adversaries. So what malware attackers do is that they build new versions of their malware then they check if it's detected by the biggest antivirus players. And then if it's detected by our AV engines, what they do is they keep building new versions of their malware until it's undetected. And then once it's undetected, they attack or customers with it and then repeat. So this is been the cat and mouse game that we've been in for years, for 10 years at least. Now, what really changed about six years ago is that we put most of our protection into our cloud protection platform. So if they actually want to check again, so like our full protection, and especially our machine learning protection, they have to be internet connected so they can communicate with a real time Cloud Machine Learning protection service.Geoff McDonald:And what this means is if they want to test their malware against our defenses before they attack our customers, it means that they're going to be observable by us. So we can look at our cloud protection logs and we can see, "Hey, it looks like someone is testing out their attack against our cloud before they attack our customers." So it makes them observable by us because they can't do it in a disconnected environment. Originally, when we came out with cloud protection, it seems like the adversaries were still testing in offline environments. Now we've gotten to the point where so many of the advanced adversaries as well as commodity adversaries are actually pre-testing their attacks against our cloud defenses before the attack our customers. And this introduces a whole bunch of adversarial ML and defensive strategies that we're deploying in order to stay ahead of them and learn from their attacks even before they attack our customers.Geoff McDonald:So we have a lot of machine learning and data science where we're really focused on preventing them from being able to effectively test with our cloud as a way to get an advantage when attacking customers. So that's one that we have a lot of work going into right now. A second thing that I really worry about for the future, this is like the really long term future, hopefully it won't be a problem for at least another decade or two or even hopefully longer. But having reinforcement learning, if we have some big breakthroughs, where we're able to use reinforcement learning in order to allow machine learning to learn new attacks by itself and carry out attacks fully automated by itself by rewarding it.Geoff McDonald:Luckily, right now, our machine learning or reinforcement learning state of the art is not anywhere close to the technology that would be needed to be able to teach an AI agent to be able to learn new attacks automatically and carry them out effectively. At least nowhere close to the effectiveness of a human at this point. But if we get to the level of effectiveness where we can teach an AI to come up with and explore new attack techniques and learn brand new attack techniques and carry out the attacks automatically, it could change the computing world forever, I think. We might be almost going back to the point where we have to live on disconnected computers or extremely isolated computers somehow but it would be kind of like a worst case scenario where machine learning has allowed the attackers to get to the point where they can use AI to automate everything and learn new attack techniques, learn new exploits, and et cetera, entirely by itself which would be a humongous problem for defensiveness.Geoff McDonald:And there's a lot of ongoing research in this right now but it's very much on the defensive side where, "Hey, we're going to use reinforcement learning to teach an attacker so that we can learn from defending against it automatically." That hypothesis is great but it's been created with the goal of trying to improve our defenses. But actually, it's also building the underlying methods needed in order to carry out attacks automatically by itself. And I think if we get to that point, it's a really big problem for security. It's going to revolutionize the way computer security works.Nic:Well, hopefully, Jeff, you and your colleagues remain one or two steps ahead in that particular challenge?Geoff McDonald:Yeah, we will.Nic:I hope you share that goal. Jeff, what are you and your team doing to make sure that you stay ahead of your sort of adversarial counterparts that are looking to that future? What gives you hope that the security researchers, the machine learning engineers, the data scientists are, hopefully, multiple steps ahead of adversaries out there?Geoff McDonald:I think our adversary situation is much better than it used to be back in the day. Back in the day, they'd be able to fully test our defenses without us even being able to see it. And now that we've forced them into the game of evading our cloud protection defenses, it allows us to observe them even before they attack our customers. So the defenses we have in place that we've already shipped as well as a lot of what we have planned is really going to be a real game changer into the way that we protect our customers where we can actually protect them even before our customers are attacked. So we're in a much better defensive situation since we're able to observe them before the attack our customers nowadays.Natalia:Thank you, Jeff, for joining us on today's show. As always, it was fantastic chatting with you and like Nick said, definitely need to have you back on the show.Geoff McDonald:Thank you very much. really love being on here.Natalia:Well, we had a great time unlocking insights into security from research to artificial intelligence. Keep an eye out for our next episode.Nic:And don't forget to tweet us @MSFTsecurity or email us at with topics you'd like to hear on a future episode. Until then, stay safe...Natalia:Stay secure. See for privacy and opt-out information.
How ready is your corporate security team to handle AI and ML threats? Many simply don’t have the bandwidth or don’t see it as a priority. That’s where security engineers like Microsoft’s Andrew Marshall step in. In this episode, hosts Nic Fillingham and Natalia Godyla speak with Andrew about just what his team is doing to teach security professionals and policy makers about the dangers of AI and ML attacks, and walks through some of the documentation, available for free online, that can help guide the response. Plus, why he really, really doesn’t want to talk about Windows Vista.   Nic and Natalia then explore what it’s like to hunt down threats with Sam Schwartz, a program manager with Microsoft Threat Experts. She came to Microsoft right out of college and didn’t even know what malware was. Now, she’s helping coordinate a team of threat hunters on the cutting edge of attack prevention.   In This Episode, You Will Learn:   • Why data science and security engineering skills don’t necessarily overlap • How attackers are using ML to change decision making • What security teams are doing to protect AI and ML systems • How threat hunters are tracking down the newest security risks • Why Microsoft Threat Experts are focused on human adversaries, not malware   Some Questions We Ask:   • What does the ML landscape look like at Microsoft? • How are ML attacks evolving? • What is ‘data poisoning’? • Why do threat hunters need to limit the scope of their work? • What skills do you need to be a security program manager? Resources Threat Modeling AI Systems and Dependencies Andrew’s LinkedIn: Sam’s LinkedIn:’s LinkedIn: Natalia’s LinkedIn: Security Blog: Transcript(Full transcript can be found at Fillingham:Hello, and welcome to Security Unlocked, a new podcast from Microsoft, where we unlock insights from the latest in news and research from across Microsoft security engineering and operations teams. I'm Nic Fillingham.Natalia Godyla:And I'm Natalia Godyla. In each episode, we'll discuss the latest stories from Microsoft security, deep dive into the newest threat intel research and data science.Nic Fillingham:And profile some of the fascinating people working on artificial intelligence in Microsoft security. If you enjoy the podcast, have a request for a topic you'd like covered, or have some feedback on how we can make the podcast better...Natalia Godyla:Please contact us at, or via Microsoft Security on Twitter. We'd love to hear from you.Nic Fillingham:Hello, Natalia, welcome to episode seven of Security Unlocked. How are you?Natalia Godyla:I'm doing well. Refreshed after Thanksgiving break. What about yourself? Did you happen to eat the pork that you were planning? Those bratty pigs?Nic Fillingham:The bratty pigs actually have survived for another day. They live to tell another tale to eat more of my home delivered fresh produce, but we did eat a duck that we farmed on our farm. So that's the second time we've been able to enjoy some meat that we grew on the farm, the little mini farm that we live on, so that was pretty exciting.Natalia Godyla:Yeah. That's been the goal all along, right? To be self-sustaining?Nic Fillingham:To some degree. Yeah. So we achieved that a little bit over Thanksgiving which was cool. How about you, what'd you do over your Thanksgiving break?Natalia Godyla:Well, I made apple bread. Oh, there's a special name for the Apple bread, but I forgot it. Pull-apart Apple bread. And I spent a lot of time on WolframAlpha.Nic Fillingham:You spent a lot of time on WolframAlpha? Did your firewall break and it was the only accessible website? How did you even get to WolframAlpha?Natalia Godyla:WolframAlpha is like Sporcle. It's like if you have time, you get to play with their technology and they've got...Nic Fillingham:Sporcle? Sorry, Sporcle?Natalia Godyla:What? Do you not know Sporcle?Nic Fillingham:I'm really old. You'll have to explain that one to me. Is this a millennial thing?Natalia Godyla:Wow. Okay.Nic Fillingham:Bring me up to speed on Sporcle.Natalia Godyla:Sporcle is like fast, quick trivia games that you play with a group in one person just types in the answers while you're running through it.Nic Fillingham:I thought it was when you take a fork and a spoon and you dip them in glitter. Anyway, so you're on Sporcle, and you're like, "I've completed Sporcle. What's next?"Natalia Godyla:And you go to WolframAlpha. That's the next step?Nic Fillingham:So, what did you pose to WolframAlpha?Natalia Godyla:All right, at what point does a cat's meow become lethal to humans? Good question, right?Nic Fillingham:At what point does a cat's meow become lethal to a human? When it's connected to a flame thrower? When the meow is a series of poison darts? What does that mean?Natalia Godyla:There are a lot of use cases for cats. In this one, it's how high the decibel of their meow is, because that can eventually hurt a human. But it's really about spacing. Where you put the cat is very critical.Nic Fillingham:The question was how loud can I make a cat's meow, so that it hurts a human?Natalia Godyla:A well-trained army of cats that meow at the exact same time, synchronized cats.Nic Fillingham:Oh, a synchronized army of cats, all directed at a single person. Would their collective uber meow, would that serve as a rudimentary weapon? That was your question?Natalia Godyla:Yes.Nic Fillingham:And? Answer?Natalia Godyla:Theoretically, but it depends on how far away all the cats end up being. I'm now thinking that I should have just like planned to capture the cat's meows in a can or something.Nic Fillingham:Capture them in a can. What does that mean?Natalia Godyla:Like a can of whoopass.Nic Fillingham:Who would capture a cats meow in a can? Okay, Professor Farnsworth.Natalia Godyla:You can tell I'm not the expert on these podcasts.Nic Fillingham:So hang on, did you work out how many cats you needed in a single location to produce a loud enough meow to hurt somebody? Do you know the answer to this question?Natalia Godyla:No. No. I was more focused on total and I don't also know the answer to the question.Nic Fillingham:All right, to all the mathematicians out there and audiologists who have dual specialties into the capturing of cat meows into cans, and then the math required to multiply them into a focused beam of uber meow as a rudimentary weapon, please send us an email, Oh, Oh, segue, segue. We have email, we have email. We got messages from people who've been listening to the show and they send some very nice things, which is great. And they also gave us some topics they would like us to cover on the show, and we're going to cover one of them today.Nic Fillingham:Shout out to Ryan and to Christian and to Tyler who all asked us to continue to thread on adversarial ML and protecting AI systems. We're doing that exactly today on this episode. We have Andrew Marshall joining us, who is going to absolutely continue to thread that Sharon Xia started a couple episodes back talking about protecting AI systems in the MDDR report, and then who are we talking to Natalia?Natalia Godyla:Sam Schwartz. So she is a security PM at Microsoft and works directly with the Microsoft Threat Experts Team to deliver managed services to our customers. So she helps to provide threats Intel back to customers and is working on scaling that out, so that more and more customers can benefit from the billions of signals that we have, that we then apply to the data that we get from customers, in order to help identify threats. On to the podcast.Nic Fillingham:Welcome to the Security Unlocked Podcast, Andrew Marshall. Thank you for joining us. Andrew Marshall:Thank you. It's great to be here. Appreciate you having me on today.Natalia Godyla:Yeah, definitely. So why don't we start off by chatting a little bit about your role at Microsoft. Can you let us know what your day to day looks like?Andrew Marshall:Sure. So I'm a Principal Security Program Manager in the Customer Security and Trust Organization at Microsoft. My role is a little bit different from a lot of people who are security engineers. I'm not part of a product group. Instead, I work across the company to solve long-tail security engineering problems that maybe one particular group may not have the authority to lead all up. So I do a variety of different things, like killing off old cryptographic protocols, where we have to bring the entire company together to solve a problem.Andrew Marshall:And lately, I'd say the past two or three years in particular, my focus has been AI and ML. In particular, the security issues that are new to the space, because it brings an entirely new threat landscape that we have to deal with. And we have to do this as an entire company. So it was another one of those cross-company security engineering challenges that I really enjoy to tackle.Natalia Godyla:And what does the ML landscape look like in Microsoft? So if it's cross-company how many models are you looking at? How many different groups are using ML?Andrew Marshall:It's a really all over the place. And by that, I mean everybody's using it. And it really is pretty much in universal usage across the engineering groups. And while there's been a big focus to everybody, whether it's in Microsoft or elsewhere, everybody's been interested in jumping on this bandwagon. But as the past couple of years, we've started to see that there are specific security issues that are unique to AI and machine learning, that we're only now, as an industry, are starting to see come out of the world of research-driven, proof of concept contrivances, where somebody created a research paper and a vulnerability that they had to make a bunch of leaps to justify. The pivot is occurring now from that into actual weaponized exploitation of these attacks.Andrew Marshall:So what we're trying to solve here from a security perspective is with this worldwide rush to jump on the AI and ML bandwagon, what is the security debt around that? What are the new products and features and detections and mitigations that we need to build as a company to solve these issues for ourselves and for the world? One of those things is really focused on education right now, because we've published a series of documents that we made, we can publish them externally. We've got a machine learning threat taxonomy, which covers the intentional and unintentional threats that are specific to machine learning. We've got some documents that were built on top of that. One of which was called Threat Modeling AI/ML Systems and Dependencies.Andrew Marshall:And this is a foundational piece of security engineering education work that's being used at Microsoft right now. The issue being security engineers, who have been... you can be a great security engineer, with tons of experience. You could have been doing this for 15 years, or more, but it most likely also means you don't have any data science expertise, or familiarity. So security engineers and data scientists are not two skillsets that often overlap. Ann Johnson calls them, "platinum unicorns", because that's just this mythical creature that nobody really seems to see. But the idea here is that we want all of our security engineers across the company to be intimately familiar with these net new security threats, specific to AI and ML.Andrew Marshall:But here's the problem with all of that. This is such a nascent field, still, especially machine learning specific InfoSec, that if you are going to address these problems today, what you need is you need automation. You need new development work to be able to detect a lot of these attacks, because of the way that they occur. They can either be direct attacks against our model, or they can be attacks against the data that is used to create the model. The detections are very primitive, if they exist at all, and the mitigations are very bespoke. So that means if you find a need to mitigate one of these machine learning threats right now, it means you're probably going to have to design that detection or that mitigation specific to your service in order to deal with that issue. That's not a scalable solution for any company.Andrew Marshall:So where we need to be is we need to get the detections and mitigations for these machine learning specific threats, get them to be transparent, on by default, inherited by the nature of using the. Platform where it just works under the hood, and you can take it for granted, like we take for granted all of the compiled in threat mitigations that you get when you build code in Visual Studio. So for example, Visual Studio, if you build code there, you inherit all of these different compiled in threat mitigations. You don't have to be a security engineer or know anything about this stuff, but you get all of that goodness just by nature of using the platform. It's on by default and you're oblivious to it. And that makes it easy to use. So, that's where we need to get with this threat landscape too. That's just a very exciting, very challenging space to be a part of.Nic Fillingham:Well, I think we're done. Thanks very much, Andrew. No, joking. Wow, so much there. Thank you for that intro. So I think my first question is this conversation we're having is following one that we have with Sharon Xia recently talking about the machine learning insecurity section that was in the recently published Microsoft Digital Defense Report. You're referring to the threat modeling AI systems and dependencies work that's up on the docs page. We'll put a link to that in show notes. When we spoke to Sharon, she really called out upfront, and I think you've just really emphasized that the sort of awareness... This is a very nascent topic. And especially at the customer level, awareness is very low and there needs to be awareness in this field. So I think what is Microsoft doing... First, maybe what is Microsoft's role in promoting awareness of this new category and what are we doing there?Andrew Marshall:So we have a role on a couple of fronts here, both within the company and more broadly, within industry and with different governments and customers around the world. So our responsibility is to act... Internally, we'll help shaping not only the educational efforts within the company, but also the research and engineering investments that are made in order to address these issues and solve these problems in this space. There's a policy shaping side of that as well, which is working with governments and customers around the world to help them shape meaningful, actionable policy. That policy in any kind of space can be a dumping ground for good intentions. So whenever people are working on some kind of new policy or some kind of new standard, we always want to make sure that everything is as practical and as actionable as it can be with... And has to be really crisp because you can't have ambiguous goals. You have to have exit criteria for all of these things.Andrew Marshall:And the reason I'm elaborating on that is because my team in the company owns the security development lifecycle. And we're very, very careful about new security requirements that get introduced into that so much so to the point that we try not to introduce new security requirements there, unless we've got some kind of automation already ready to roll for people to use. And that way, we can just tell them, "Hey, this is now a mandatory thing that you have to do, but it's really just run this tool and fix these errors. It's not some kind of new manual attestation or big painful exercise to go through." And that's how we can keep adapting the SDL policy. On the responsible AI side and AI and ethics, we've got... This responsible AI standard that we're working on is basically the guiding principles around responsible AI for Microsoft in terms of how we deal with bias and fairness and transparency and reliability and safety issues as they relate to AI, as well as to security. And this is another element of policy that's being shaped within the company.Nic Fillingham:So you mentioned that very few of these guidances have been automated. Obviously, one of the goals is probably, I assume, to get them automated into toolsets and into SDL. So let's... I'm going to put a customer hat on. I'm a customer of Microsoft. How should I feel about the work that Microsoft is doing to secure its own AI and ML systems? So obviously, we're practicing what we preach here and putting these guidances into place. How is success being measured? Or what are the metrics that we're using to, be it manually or automated, to make sure that our own AI and ML systems are protected?Andrew Marshall:We're spinning up significant research and engineering investments across the company specifically to tackle these kinds of problems. Part of that is largely security. And it's part of this broader series of AI and ethics investments that we're making, but the security issues in particular, because we know that we've got customers reporting these kinds of things, and because we know that we've got our very own specific concerns in this space, we're putting together roadmaps to deal with these kinds of issues as specific sets of new product features and threat detections and mitigations in this space.Andrew Marshall:We understand that you can't catch any of these things manually. It takes automation to catch any of this stuff. So that gives us a roadmap of engineering investments that we can prioritize and work directly with engineering groups across the company to go solve that. And the idea here being that when we deliver those solutions, they're not just available to Microsoft services, but they'll be made available to customers of Microsoft as well.Natalia Godyla:So, Andrew, how are we seeing these attacks start to evolve already? So if you could talk us through a couple of examples, like data poisoning, that would be awesome.Andrew Marshall:Oh, I'd love to. So data poisoning is something that we've seen our own customers impacted by because as we point out in our threat modeling guidance, there's a tremendous over-reliance on using public uncurated data feeds to train machine learning models. Here's an example of a real situation that did happen. So a customer was aggregating trading data feeds for a particular futures market. Let's just say it was oil. And they're feeding these training data feeds from different trading institutions, brokerages, or trading websites or whatever. They're taking all this stuff over a secure channel, they're generating a machine learning model from it. And then they're using that ML model to make some really high consequence decisions like is this location a good place to drill for oil or bid on rights by which you can drill for oil? Or do we want to take a long position or a short position in the oil futures market?Andrew Marshall:So they're essentially trusting the decisions that come out of this machine learning model. And what's the nature of futures trading data feeds there's new data every day, so they're constantly incorporating this new data. Talking about the blind reliance on this untrusted data, even though it was over a secure channel, one of the training data providers was compromised not in a way that resulted in the website being shut down, but what happened was their data was contaminated. The data that they were sharing with everybody else. Unless you're actively monitoring for something like this as the provider of that data, there's no way that you're going to know that you're sending out that data to everybody else.Andrew Marshall:So if the providers are unaware of the compromise, then the consumer of the data is also going to be equally as oblivious to the fact. So what happens is over time, that data became trusted high confidence garbage within that trading data model. So then that led to these decisions like drilling for oil in the wrong place or longing the futures market when they should have been shorting it and vice versa. So the point here is without automation to detect that kind of data poisoning attack, you don't know anything went wrong until it blows up in your face.Natalia Godyla:It really gives you perspective because I feel like normally when you're hearing about cyber attacks, you are hearing about data being stolen and then sold or money itself being stolen. But in the case that you just explained, it's really about altering decision-making, it wasn't just direct money stealing.Andrew Marshall:That was an interesting case because we're also thinking, all right, well, was it a targeted attack against the consumer or the people building machine learning models? How did the attacker know that? Were they looking to see what kind of outcomes this would generate? Is this the only place that they were doing that? Of course the data provider doesn't know. That's one of the more interesting, more insidious attacks that we've seen because we've got to create new types of tools and protections in order to even detect that kind of stuff in the first place. So you're looking for... As your machine learning models are being built, you're looking at taking on new data and looking for statistically significant drift in certain parts of the data that deviate from what looks normal and the rest of your data, and we're looking at ways of solving that. And that's an interesting space. So, yeah.Natalia Godyla:So you noted that one of the potential reasons that the threat actor was playing around with that ML system for that customer example was because they were also just trying to figure out what they could do. So if it's so nascent then threat actors, are they in a similar place as us? Are they ahead of us?Andrew Marshall:Well, we've already had that pivot from contrived research exploits where people are just trying to show off. We've already had that pivot into actual exploitation. So I don't know how to go back and attribute the level of attacker sophistication there. I don't think it was actually... In the attack that I mentioned here, the oil company scenario, that was compromised through traditional security vulnerabilities of that data provider. And I think the jury is still out on the final attribution of all of that, as well as the level of attacker sophistication or if... What would be even more interesting than all of that is really what other customers of that data provider were compromised in this and building machine learning models that were contaminated by that data. Think about hedge funds, who else was compromised by this and never even found out? Or who else had a model blow up in their face? That'd be a very interesting thing to see.Nic Fillingham:The question I wanted to wrap up with, Andrew, is make me feel like we're on a good path here. Like, can we end on a high note? We talked about a lot of very serious scenarios and the consequences for adversarial ML. And obviously it's very important and very nascent, but should I feel like the good guys are winning? Should I feel like we've got good people on this? We're making great progress? That we should feel confident in AI and ML systems in-Andrew Marshall:Yeah, absolutely.Nic Fillingham:The second half of 2020?Andrew Marshall:That's our entire focus with the AI and ethics and engineering and research group. We are bringing the entire weight of Microsoft to bear around these issues from a research, engineering, and policy perspective. And we want to solve all these issues so that you do have trustworthy interactions with all of our products. And that's an essential thing that we realized collectively as a company that has to happen where people won't use these kinds of products. If it doesn't generate an outcome that you can trust is going to be accurate and free of bias and something that you can rely on, then people just won't use those things. So we've got the AI and security centers of gravity working across the company with research and policy experts to tackle these issues. It's a fascinating time to be a part of this. I think that... I just had my 20 year anniversary last month, and I think this is about the most fun I've had period in the past 20 years working on this stuff now.Nic Fillingham:It wasn't the launch of Windows Vista?Andrew Marshall:I have so many horror stories from that. We really don't want to air those.Nic Fillingham:Well, that's awesome. Gosh, what was I... I had this great question I was going to ask you and then the Vista joke popped in and now my brain is mulched.Natalia Godyla:I love how that took priority.Nic Fillingham:Like the most intelligent question I'm going to ask the entire interview and it's like just a joke bonk.Andrew Marshall:I have some very, very funny stories from Vista, but none that are appropriate for here.Nic Fillingham:Well, we may have to bring you on another time, Andrew, and try and sanitize some of those stories because the statute of limitations has surely run out on having to revere every single release of Windows. Surely we can make fun of Vista soon, right?Andrew Marshall:I'm sure we can.Nic Fillingham:So, Andrew, final question, where do you recommend folks go to learn more about this space and keep up to speed with any of the advancements, new taxonomy, new guidelines that come out?Andrew Marshall:I would definitely keep tabs on the Microsoft Security blog. That's going to be the place where we drop all of the new publications related to anything in this space and connect you with security content more broadly, not just AI and ML specific, but yeah, the Microsoft Secure Blog, that's where you want to be.Nic Fillingham:Great. Thanks Andrew Marshall for your time. We'll also put a link up to the guidelines on the doc's page.Andrew Marshall:All right. Thank you very much for having me today. It's been great.Natalia Godyla:And now let's meet an expert in the Microsoft Security Team to learn more about the diverse backgrounds and experiences of the humans creating AI and tech at Microsoft.Natalia Godyla:Hello everyone. We have Sam Schwartz on the podcast today. Welcome Sam.Sam Schwartz:Hi, thanks for having me.Natalia Godyla:It's great to have you here. So you are a security PM at Microsoft. Is that correct?Sam Schwartz:That is correct.Natalia Godyla:Awesome. Well, can you tell us what that means? What does that role look like? What is your day-to-day function?Sam Schwartz:Yeah, so I support currently a product called the Microsoft Threat Experts and what I am in charge of is ensuring that the incredible security analysts that we have, that are out saving the world every day, have the correct tools and processes and procedures and connections to be the best that they can be.Natalia Godyla:So what are some of those processes look like? Can you give a couple examples of how you're helping to shape their day to day?Sam Schwartz:Yeah. So what Microsoft Threat Experts does, is it as a managed threat hunting service provided by Microsoft defender ATP product and what they do is our hunters will go through our customer data in a compliant safe way, and they will find bad guys, human adversaries inside of the customer telemetry. And then they notify our customers via a service called the targeted attack notification service.Sam Schwartz:So we'll send an alert to our customers and say, "Hey, you have that adversary in your network. Please go do these following things. Also, this is the story about what happened, how they got there and how you can fix it."Sam Schwartz:So what I do is I try to make their lives easier by initially providing them with the best amount of data that they can have when they pick up an incident.Sam Schwartz:So when they pick up an incident, how do they have an experience where they can see all of the data that they need to see, instead of just seeing one machine that could have potentially been affected, how do they see multiple machines that have been affected inside of a single organization? So they have an easier time putting together the kill chain of this attack.Sam Schwartz:So getting the data and then also having a place to visualize the data and easily make a decision as to whether or not they want to tell a customer about it, does it fit the criteria? Does it not? Is this worth our time? Is this not worth our time? And then also providing them with a path to, with that data quickly create an alert to our customers so that they know what they're doing.Sam Schwartz:So rather than our hunters, having to sit and write a five paragraph essay about what happened and how it happened, have the ability to take the data that we already have, create words in a way that are intuitive for our customers, and then send it super quickly within an hour to two hours of us finding that behavior.Sam Schwartz:So all of those little tools and tracking, and metrics and easier, like creating from data, creating words, sending it to the customers, all of that is what I plan from a higher level to make the hunters be able to do that.Nic Fillingham:And to better understand the scale of what's happening here, like with a typical customer, what is the volume of signal or alerts or, I'm not sure what the correct taxonomy is, but what's the volume of stuff that's being monitored from the customer and then is being synthesized down to a bunch of alerts that then go and get investigated by a hunter?Sam Schwartz:So I don't have a per customer basis, but we have about, I think it's either 450 customers currently enrolled in our program. And unfortunately, we can't take everyone that would like to join us. Our goal is that we will eventually be able to do that, but we don't have enough people and we're still building our tooling to allow us to scale.Sam Schwartz:So with our 450 customers, we have every month, about 200,000 incidents that get created and we then bring that down. So some of those incidents don't get investigated because they don't meet our bar. Some of those incidents get investigated, but aren't interesting enough to actually have an alert created. And some of them even, although the alert is created, it's not actually interesting enough to send, or we've already sent something similar and it's not worth it.Sam Schwartz:So from those 200,000, we send about 200 to like 250 alerts a month about, but it also depends on the landscape. Like it depends on what's going on that-Nic Fillingham:And if I go even higher up the funnel, so before the 200,000 is it, what's the right taxonomy, is it an alert?Sam Schwartz:Incidents. We call them incidents.Nic Fillingham:... What's above an incident. What is, because I assume it's just tons and tons and tons of network logs and smaller signals that end up getting coalesced into an incident. Is that correct?Sam Schwartz:Yeah. So what we do is we call them traps. So what they are is they're queries that run over data that finds something super interesting. And you can think about these as similar to alerts that customers get, but much, much, much lower fidelity.Sam Schwartz:So for us, for our products, a trap, if it fires a hundred times and of that a hundred times, 99 of them are false positives, 99% of them are not super helpful for the customer, we're not going to send that to the customer. That's bothering them 99 times that they don't need to be bothered. But for our service, our whole thing is that we are finding that 1% that our customer doesn't know about.Sam Schwartz:So we have extremely low fidelity traps. Some of them are high fidelity that it can run a thousand times and only one time is it important? We want to see every a thousand times because that one time is worth it. So we have traps, I think we have about 500 of them. Some of them return thousands of results a day. Some of them won't return results for months.Sam Schwartz:And if that gets a hit, then those are the things that get bubbled up into our incidents. We cluster all of those trap results into the incidents, so that's ensuring that our hunters get all the information that they need when they log on, so the signals are massive. There's a massive amount. I don't even have a number.Natalia Godyla:I have literally so many questions.Sam Schwartz:Oh my God, happy to help.Natalia Godyla:So you said earlier, there's a bar for what the Microsoft Threat Experts will focus on. So what is in scope for them? What meets the criteria?Sam Schwartz:We are focusing on human adversaries. So we're not focusing much on commodity malware, as much as we are focusing on a hands-on keyboard attacker. So there are some traps that are, some of them are commodity malware, but paired with other traps so paired with other signals, that could be a hands-on keyboard person. And those are things we look at, but then maybe some of the traps on their own don't meet a bar for us to go look at.Nic Fillingham:Is that because commodity malware is basically covered by other products, other services?Sam Schwartz:(Affirmative). It's covered by our defender ATP product in general. So our hunters wouldn't be adding. Our whole point is that we have hunters who are adding context and value to the already incredible ATP product. And since ATP is already alerting and covering that, we'd rather find the things that aren't being covered.Nic Fillingham:So Sam, let's go back in time a little bit, so tell us about how you found yourself in the security space and maybe it's a separate story maybe it's the same story and how you got to Microsoft. We'd love to learn your journey, please.Sam Schwartz:It is the same story. Growing up, I loved chemistry.Nic Fillingham:That's too far back.Sam Schwartz:I know.Nic Fillingham:Oh, sorry. Let's start there.Sam Schwartz:I loved Chemistry. I loved like molecules and building things and figuring out how that all works. So when I went to college, I was like, I want to study chemical engineering. So I through my education became a chemical engineer, but I found that I really liked coding. We had to take a fundamentals class at the beginning and I really enjoyed the immediate feedback that you got from coding. Like you did something wrong, it tells you immediately that you messed up.Sam Schwartz:And also when you mess up and you're super frustrated and you're like, why didn't this work? Like I did it right. You didn't do it right, it messed up for a reason. And I really liked that. And I thought it was super interesting. And I found myself like gravitating towards jobs that involved coding.Sam Schwartz:So I worked for Girls Who Code for a summer. I worked for a Dow Chemical Company, but in their robotics division. So I was still like chemical engineering, but I got to do robots. And then when I graduated, I was like, I think I want to work in computer science. I don't like this chemical engineering. It was quite boring, even though they said it would get more fun, it never did. We ended up watching water boil for a lot of my senior year of college. And I was like, I want to join a tech company.Sam Schwartz:And I looked at Microsoft and they're one of the only companies that provide a program management job for college hires. So a lot of PM positions because there's a lot of high level thinking, coordinating and collaboration. A lot of PM positions are one of those, like you need experience, but in order to get experience, you have to do the job and it's like one of those weird circles and Microsoft allows college hires to do it.Sam Schwartz:So when I interviewed, I was like, I want to be a PM. It sounds fun to get to hang out with people. And I ended up getting the job, which is awesome.Nic Fillingham:Is that all you said in the interview? Just, it sounds fun to get to hang out with people?Sam Schwartz:Yes. I was like, this is it, this is my thing. What they did is they, in my interviews, they asked me a bunch of, they asked me a very easy coding question, I was so happy. I was so nervous that I wasn't going to get a pass that one, but that was easy. And then they asked me a design question. They asked me, "Pick your favorite technology." And me, I'm sad to say it. I feel like I'm better now looking back on myself, but I'm really not good with technology in general.Sam Schwartz:So they're like pick your favorite technology. And I was like, I'm going to pick a chemical engineering plant because I didn't know anything. So I picked an automation plant as my favorite technology. And they asked me a lot of questions around like, who are the customers? What would you do to change this to affect your customers? Who gets changed? How would you make it better?Sam Schwartz:Then I was talking specifically about a bottling plant, just because that's easy to understand. And I left that interview and my interviewer was like, I didn't know, he said, "I didn't know anything that you were talking about, but everything you said made perfect sense because it's about how can you take inputs, do something fun and then have an output that affects someone. And that's everything that we do here. Even though it's a bit obfuscated and you have a bunch of data and bad guys and hunters hunting through things, it's taking an input and creating something great from it."Sam Schwartz:And that's what we learned in our chemical engineering world. And I ended up getting this job and I walked on my first day and my team and they're like, "You're on a Threat Intelligence Team." I was like, "What does that mean?" And-Nic Fillingham:Oh, hang on. So did you not know what PM role you were actually going to get?Sam Schwartz:No. They told me that I was slated for the Windows. I was going to be on a Windows team. So in my head like that entire summer, I was telling people I was going to work on the start button just because like, that's what... I was like, "If I'm going to get stuck anywhere, I'm going to have to do the start button. Like that's where my-"Nic Fillingham:That's all there is. Windows is just now a start button.Sam Schwartz:I was like that what... I was guaranteed, I'm going to get the star button or like Paint. Actually, I probably would have enjoyed Paint a lot, but the start button and I came and they were like, "You're on Threat Intelligence Team." And I was like, "Oh, fun." Sam Schwartz:And it was incredible. It was an incredible start of something that I had no idea what anyone was talking about, when they were first trying to explain it to me in layman's terms, they're like, oh, well, there's malware and we have to decide how it gets made and how we stop it. And I was like, what's malware? I was like, you need to really dumb it down, I have no idea what we're talking about. And initially when I started on this threat intelligence team, there were only five of us. So I was a PM and they had been really wanting a PM, and apparently before they met me were happy to get a PM, but weren't so happy it was a college hire. They're like-Nic Fillingham:Who had never heard of malware.Sam Schwartz:We need structure.Nic Fillingham:And thought Windows was just a giant anthropomorphic start menu button.Sam Schwartz:They're like, we need structure, we need a person to help us. And I was like, hi, nice to meet you all. And so we had two engineers who were building tools for our two analysts and it was, we called ourself a little startup inside of security research inside of the security and compliance team, because we were figuring it out. We were like, threat intelligence is a big market, how do we provide this notion of actionable threat intelligence? So rather than having static indicators of compromise, how do we actually provide a full story and tell customers to configure, to harden their machines and tell a story around the acts that you take to initiate all of these. These configurations are going to help you more than just blocking IOCs that are months old. So figuring out how to best give our analyst tools, our TI analysts, and then allow us to better Microsoft products as a whole.Sam Schwartz:So based on the information that our analysts have, how do we spread that message across the teams in Microsoft and make our products better? So we were figuring it out and I shadowed a lot of analysts and I read a lot of books and watched a lot of talks. I would watch talks and write just a bunch of questions. Then finally, as you're around all these incredibly intelligent security people, you start to pick it up, and after about a year or so I would send meetings and I would listen to myself speak and I was like, did I say that? Was that me that one, understood the question that was asked of me and then also was able to give an educated answer? It was very shocking and quite fun. And I still feel that way sometimes, but I guess that's my journey into security.Natalia Godyla:Do you have any other suggestions for somebody who is in their last years of college or just getting out of college and they're listening to this and saying, heck yes, I want to do what Sam's doing. Any other applicable skills or tricks for getting up to speed on the job?Sam Schwartz:I think a lot of the PM job is the ability to work with people and the ability to communicate and understand what people need and be able to communicate that in a way that maybe they can't communicate. See people's problems and be able to fix them. But I think a lot of the PM skills you can get by working collaboratively in groups, and that you can do that in jobs, you can do that in classes. There's ample opportunity to work with different people, volunteering, mentoring, working with people and being able to communicate effectively and connect to people and understand, be empathetic, understand their issues and try to help is something that everyone can do and I think everyone can be an effective PM. On the security side, I think reading and listening. Even the fact that, the hypothetical was someone listening to this podcast that are already light years ahead of I was when I started, but just listening, keeping up to date, reading what's going on in the news, understanding the threats, scouring Twitter for all the goodness going on.Sam Schwartz:That's the way to stay on top.Nic Fillingham:Tell us about your role and how you interface with data scientists that are building machine learning models and AI systems. Are you a consumer of those models and systems? Are you contributing to them? Are you helping design them? How do you fit into that picture?Sam Schwartz:So a little bit of all of the things that you mentioned, being a part of our MTE service, we have so many parts that would love some data science, ML, AI help, and we are both consumers and contributors to that. So we have data scientists who are creating those traps that I was talking about earlier for us, who are creating the indicators of malicious anomalous behavior that our hunters then key off of. Our hunters also grade these traps. And then we can provide that back to the data scientists to make their algorithms better. So we provide that grading feedback back to them to have them then make their traps better. And our hope is that eventually their traps, so these low fidelity signals, become so good and so high fidelity that we actually don't even need them in our service, we can just put them directly in the product. So we work, we start from the incubation, we provide feedback, and then we hopefully see our anomaly detection traps grow and become product detections, which is an awesome life cycle to be a part of.Nic Fillingham:I want to change topics then, but this one's going to need a little bit of context setting because you are famous inside of Microsoft for anyone that has completed one of our internal compliance trainings. I don't even know how to describe this to people that haven't experienced it. Natalia, we've both done it. So there's this thing at Microsoft called Standards of Business Conduct, it's like a internal employee compliance. This is how you should behave, this is how you should function as a responsible employee and member of the Microsoft family, but then also how we work with customers and everything. And it's been going on for a few years. Sam, you had a cameo, you were the only non-professional actor in the recent series, that's correct?Sam Schwartz:I was, I was, I'm famous, I will be signing headshots when we're all back in the office.Nic Fillingham:So tell us about how did this happen?Sam Schwartz:So I, as anyone who has seen the Standards of Business Conduct videos, I wouldn't call them a training, I would call them a production.Nic Fillingham:An experience. Or production.Sam Schwartz:An experience, yeah. An experience.Nic Fillingham:They're like a soap opera. It's almost like Days of Our Lives. They really stir the emotion and we get attached to these characters and they go on wild journeys in a very short space of time.Natalia Godyla:I was just watching an episode and I literally got stressed.Sam Schwartz:Yeah, you're so invested in these characters and their stories and you're rooting for them to do the right thing. And you're like, come on, just be compliant. And in my first week on the job I was telling, I watched this training as everyone who starts Microsoft has to do and I was telling my team that I was obsessed with the main character who has his own trial and tribulations throughout the entire series. And I just thought it was fun and I was like, how do I get on it? That was my thing when I first joined, how do I get on Standards of Business Conduct? And every year, Microsoft is super passionate about giving, giving back, donating money, and every October we have this thing called the Give Campaign where every employee is encouraged to give back to their community.Sam Schwartz:And one of the ways that they do is they have an auction. So some of the auction things are, you get lunch or golf with Satya, or you get assigned, I don't know, computer or X-Box from Phil Spencer or whatever it is. I made those up.Nic Fillingham:You get to be the Windows start button for a day.Sam Schwartz:You get to be the Windows start button for a day. And one of those is a cameo in Standards of Business Conduct. And you can donate a certain amount of money and there's a bid going, where the person who donates the most money is at the leaderboard and then if you donate more money, you got on top. So a silent auction before giving back and donating. And I saw that last year on the gift campaign, but I didn't think much of it. It had a high price tag and I didn't want to deal with it. And then a couple of months later, I had just gotten back from vacation and my skip level was like, hey, I missed you a lot, let's get lunch. And I was like, okay, great, I love that.Sam Schwartz:And he was like, I want to go somewhere fun, I want to go to building 35, which is the executive nice cafeteria building at Microsoft, which is not near our office. And I was like, okay, weird, he wants to go to another building for lunch, but we can go do that. So I went with him and it was five to 10 minutes into our lunch and these people come up to our table and they're like, can we sit with you? And I'm looking around and there are tons of tables, I'm like, what are these people encroaching on my lunch for? I just want to have lunch and chat and these people want to come sit at my table, but of course, we're going to let them sit at our table. And I look over at the guy who's sitting next to me and it's the main character from Standards of Business Conduct. It is the actor, it is-Nic Fillingham:It's Nelson.Sam Schwartz:It's Nelson. And I fan girled over him for a year and a half now, I've seen all his work, I'm a huge fan.Nic Fillingham:Please tell me it was a Beatles on their first tour to America moment. Please tell me there was screaming, there was fainting.Sam Schwartz:I blacked out.Nic Fillingham:That's the picture in my head.Sam Schwartz:I don't remember. I don't remember what happened because I actually blacked out. And there's a video of this and you can see my body language, when I realized you can see me grab the arms of the chair and my whole body tenses up, and I'm looking around frantically, like, what's happening. And the woman who was sitting next to my skip-level, she actually created Standards of Business Conduct and she's in a lot of the videos, her name is Rochelle. And she's like, your team has pulled together their money and bought you a cameo in our next Standards of Business Conduct. And I turned around and my entire team is on the balcony of the cafeteria filming me and it was very cute and very emotional. And I got to see Nelson and then I got to be in Standards of Business Conduct, which is awesome. And it was a super fun experience.Nic Fillingham:So in the Microsoft cinematic universe, what is your relationship to Nelson? Are you a colleague?Sam Schwartz:We're all on the same team.Nic Fillingham:So you're a colleague.Sam Schwartz:We are on the same team.Nic Fillingham:So forever, you're a colleague of Nelson's.Sam Schwartz:Yeah, I am. And he knows who I am and that makes me sleep well at night.Natalia Godyla:Thank you, Sam, for joining us on the show today, it was great to chat with you.Sam Schwartz:Thank you so much for having me. I've had such a fun time.Natalia Godyla:Well, we had a great time unlocking insights into security, from research to artificial intelligence. Keep an eye out for our next episode.Nic Fillingham:And don't forget to tweet us @MSFTSecurity, or email us at with topics you'd like to hear on a future episode. Until then, stay safe.Natalia Godyla:Stay secure. See for privacy and opt-out information.
Ever wonder why it's so difficult to really secure a network, systems or data? Cyber criminals are stepping up their game, even as security gets stronger and stronger, and they’re using all sorts of new techniques to break through enterprise walls. In this episode, hosts Nic Fillingham and Natalia Godyla speak with Donal Keating, Director of Innovation and Research for the Microsoft Digital Crimes Unit, about one of the key findings in the latest Microsoft Digital Defense Report: how attackers are adapting and becoming more sophisticated. Plus how social engineering is revealing the true weakest link in any security plan -- and it’s something you might not expect.   Then they dive into what it’s like to hunt threats with Michelle Lam, who brings fresh eyes to every security problem she faces at Microsoft. She explains why not spending time in a SOC early in her career helps her spot potential attacks others might miss, and why she’s so passionate about helping serve under-represented communities and inspiring the next generation of security professionals.   In This Episode, You Will Learn:• How cyber attackers are using the cloud • Why humans are the weakest link in every security system • The new steps cyber criminals are taking to get people to trust them • How threat hunters look for malicious activity • How networking helps young security professionals Some Questions We Ask:   • What new threat trends are emerging? • How should security professionals prepare for new threats? • What is a homoglyph? • Why is threat hunting a uniquely human-based activity? Resources  Microsoft Digital Defense Report, September 2020 Donal’s LinkedIn:’s LinkedIn:’s LinkedIn: Natalia’s LinkedIn: Security Blog: Transcript(Full transcript can be found at Fillingham:Hello and welcome to Security Unlocked, a new podcast from Microsoft, where we unlock insights from the latest in news and research from across Microsoft security, engineering and operations teams. I'm Nic Fillingham.Natalia Godyla:And I'm Natalia Godyla. In each episode, we'll discuss the latest stories from Microsoft security, deep dive into the newest threat intel, research and data science.Nic Fillingham:And profile some of the fascinating people working on artificial intelligence in Microsoft security. If you enjoy the podcast, have a request for a topic you'd like covered, or have some feedback on how we can make the podcast better-Natalia Godyla:Please contact us at or via Microsoft Security on Twitter. We'd love to hear from you. Hi Nick, welcome to another episode. How's it going?Nic Fillingham:Hi, Natalia, I'm a little angry, actually. I'm a little cranky. I don't know if I've said on the podcast before, I live on sort of a small farm, about 30 minutes East of Seattle. And we've got some farm mammals, we've got piglets, recently, they were born in the spring. And this morning the piglets found our delivery of fresh fruit and vegetables from CSA and they ate them all. They ate $75 worth of beautiful organic fruit and veggies, that was meant to last us for the next month. So I'm having pork for Thanksgiving.Natalia Godyla:Those are the brattiest pigs.Nic Fillingham:Yeah well we initially... Their names when they were born, they were super sweet and we called them June and July, my daughters called them that, but we've renamed them to Beavis and Butt-Head because they are stupid jerks.Natalia Godyla:Wow, that's harsh.Nic Fillingham:You think they listen to the podcast? I have given both of them iPhones. Apart from that I'm good, how are you Natalia?Natalia Godyla:Wow, I mean, I can't compete with that story. I'm definitely not at war with one of my piglets.Nic Fillingham:You're in Boston, Massachusetts, I think... You're not downtown, you're in more of the leafy green, sort of, oldie worldy part, aren't you?Natalia Godyla:I am, I'm near Cambridge dealing with equally bratty, but amusing animals. While I don't have the farm set up you have, I have the Somerville turkey.Nic Fillingham:The Somerville turkey? Is that a ghost of a Turkey?Natalia Godyla:Right, it sounds like the headline to a scary movie.Nic Fillingham:Yeah, it's like a turkey shaped poltergeist, what is that?Natalia Godyla:It's just the turkey that causes mayhem in our little neck of the city.Nic Fillingham:Is the turkey's name Somerville or is that the neighborhood?Natalia Godyla:Oh, that's the neighborhood.Nic Fillingham:Does the Turkey have a name?Natalia Godyla:I don't know if it deserves a name.Nic Fillingham:And what does it do, how does it cause mayhem? Is it tipping over trash cans and spray painting swear words on sides of people's houses?Natalia Godyla:I think you might be mixing up a hoodlum with a turkey. No, it blocks traffic and is a great source of distraction for everyone doing remote work in Boston right now.Nic Fillingham:I mean, because you live so close to the storied Cambridge University, I can only assume that a turkey is a much more sophisticated, intelligent turkey. And when it's blocking traffic, it's pulling out traffic cones, it's setting up fake road work, à la Ghostbusters Two.Natalia Godyla:Yeah, this was a very unexpected turn, but I'm impressed at the short number of leaps until we got to Ghostbusters.Nic Fillingham:Hey man, I can get to Ghostbusters in two leaps. Doesn't matter what the topic is. And speaking of turkeys and Thanksgiving, these two turkeys, that's you and me and Natalia, we are very thankful for our guests that joined us on episode six of Security Unlocked. First up, we continue our exploration of some of the topics in the Microsoft Digital Defense Report, the MDDR. Donal Keating is joining us to talk about the increase in sophistication in cyber attacks, and so what does that mean to have seen an increase in sophistication in cyber attacks over the last sort of 12 to 18 months? And some of the sort of high level observations that are in the report, that's a great conversation.Natalia Godyla:And we have Michelle Lim on the show today, threat Hunter at Microsoft. She'll be sharing her path to security and how industry organizations and mentorship have helped her identify new skills and interests within this security space. It's really great to hear how she's leaned on the community to help drive her career and her passions for the cybersecurity realm.Nic Fillingham:And happy Thanksgiving to everyone celebrating in North America. Everyone else happy late November, early December to you. We hope you enjoy the podcast.Nic Fillingham:Welcome to the podcast, Donal Keating.Donal Keating:Hi.Nic Fillingham:Thanks for your time. So Donal we like to start the podcast by asking our guests to give sort of a brief introduction to themselves. What's your job at Microsoft, but sort of what does that look like day to day?Donal Keating:So my role is I'm director of innovation and research for the digital crimes unit, and I generally accepted I have the best job in Microsoft. But what it really means is I sit between a group of people who have regular investigative and analytic jobs and the lawyers who take the cases that we build up and what I consider the data hacking. So we have access to lots of data, lots of crime mechanics, and it's my job, really, to figure out techniques to unveil the criminality and see if we can assist an attribution or mitigation against a particular crime. I'm just sort of the new guy on the block when it comes to new types of crime or new patterns in cyber crime.Nic Fillingham:Are you the Oracle, if I can use a DC universe analogy, or do you prefer a different... What's the superhero role that best fits what you do?Donal Keating:Glue. I'm just incredibly inquisitive glue. And I know very little and it's great having... I feel like a three-year-old going around Microsoft asking, "What does that do?" You need to be inquisitive in this.Nic Fillingham:That's sort of what Natalia and I are doing on the podcast. That sounds awesome. So Donal, thank you for joining us. In the conversation today, we want to talk about, really one of the biggest headlines coming out of the recently released Microsoft Digital Defense Report for 2020. So it's a report that came out in September. Tom Burt, who leads, I think the organization you're a part of, customer security and trust. He authored the blog post announcing the report and sort of the big takeaway... The big headline there was that, in this last period, cyber threat sophistication has increased and we've never seen it sort of this sophisticated. And so we've invited you onto the podcast today to really help us unpack this idea of cyber threat sophistication and the fact that it is increasing. So if I could start with sort of a pretty big question, cyber threat sophistication is increasing. What does that mean? How do we think about that? How do we measure that? What does it mean for folks out there to know that cyber threats are increasing in sophistication?Donal Keating:Yeah, that's a good question and the way I would... The reason, first of all, the sophistication of this cyber crime is increasing, is largely that the sophistication of the defense has increased significantly. So as more workloads run on cloud environments, operating systems become more secure. People would become more security conscious, there is just more technology in the production area. Criminals by their nature need to adapt to that challenge, so in one area and what I would call traditional hacking where people are trying to gain remote access to a device. They have pivoted away from trying to find zero day exploits and they've actually pivoted to some human engineering. Now, the human engineering may be to get the malicious workloads to run on machines to unlock them and allow malware to be installed.Donal Keating:So that's one area that they need to have got more sophisticated just to get around the defenses. But the second area of where we see sophistication, is cyber crime is now a business. And as a business, there was specialization in that business. So you have very specialized people who will develop malware, ransomware, their specialization and the distribution of that. People who have droppers, people who have networks, botnets, where they will use those botnets to do other things such as proxy attacks, enterprises, proxy attacks on other types of resources.Donal Keating:Even within that, we see a level of automation that we have not seen in the past. So what we would call machine on machine activity is certainly evidence of some of the attacks that we see. But even in the final stages of a cyber crime attack, where it comes to either the ransomware, the exfiltration of data, or just the pure stealing of money out accounts as a result of phishing. The way that money is being muled has also increased. Now, not at the same rate as the sophistication we see in the phishing lures or in the methods of getting people's credentials, because the old saying goes, people used to hack into a computer system, now they log in. A lot of what cyber criminals are doing initially on the attack, is getting some set of credentials to get onto the environment, and then do what they do best, which is do cognizance work across the organization to see more people get more credentials and basically map out the network.Natalia Godyla:With that, can you talk us through a couple examples of how these threats have changed or what new emerging trends are coming out?Donal Keating:Let me give you an example, so banks obviously need to have a significant amount of protection for people logging in, so remote banking. So there's normally a control that says from a given IP address, there can only be, for a given user, there can only be a certain number of login attempts. Now, if you're like me, that's almost guaranteed to be five login attempts because I can never remember what my password is, but I know it's some combination of something. So it is not unusual for normal behavior to be one IP address, one username, three, four, five login attempts. Therefore, any protections that the bank put in place to make sure that the people who are hacking, it needs to meet that criteria, you don't want to disable the customer. And those controls very often called shape controls, will limit the amount of traffic coming into the bank from any one IP address.Donal Keating:So I have seen a case doing what's called credential stuffing. So that's a single IP address with a single username and then multiple attempts to log in. So the attack that we saw, the bank had that control, how many attempts it had been set up at over 20 attempts per hour were allowed. And the bank realized they were having this credential stuffing attack. So what they did is they reduced the number of login attempts that were allowed. And within about an hour, this particular attack dropped down to 14 attacks per hour. Now this was not one IP address, there were 400 IP addresses per hour, probing the banking system. And as the banking system can change their controls, this network of machines adjust to their controls. They also need to do one other thing, the bank had controls as to where those IP addresses had to be located.Donal Keating:The criminals had organized a botnet to deliver the traffic via proxies only in the region where they would be accepted. So they had done two things, they had modified the rate at which they were probing the username, password combination, and they were coming from the location that they were expected to come from. In cyber crime, that's becoming quite a common pattern, that you're not getting the IP addresses from halfway around the world, the login attempts are coming from the area that you expect them to come from. It starts to become quite difficult for defenders to defend against. Now, more barriers will be put up and the cyber criminals will figure a way to get around that, but the improvement in protections and the more security that is applied, requires these cyber criminals to become more inventive in the way they do their thing.Nic Fillingham:So is that rapid agility, that ability to respond? Is that in part the sophistication increase that we're seeing the fact that, to use your example there, that those attackers were able to ascertain that the number of permitted tries per hour was reduced from 20 to 15, and the ability for them to identify that and then adjust their attack. That's in some way, what you're seeing in sophistication increase, whereas in the past, either that wouldn't have happened or it might've taken them weeks or even months to make that change?Donal Keating:Well, two things, one is they are now using cloud resources to do this. So the attack is not coming from a PC somewhere, this is a battery of VMs set up to behave in a particular way. Their ability to deploy VMs at scale, give them instructions at scale to do these things is a thing that first of all, it just wasn't available previously. But the fact that they are now using the sophistication of technology that large enterprises use to commit crime is indication to me of increasing sophistication. For instance, there are many automated systems to take down. So there's lots of defenders in the world and they see traffic coming from things that they understand are malicious. There are many, many systems to communicate that threat intelligence across companies and those things such as a URL, a malicious URL can be taken down relatively quickly. But if the domain has the ability to stand up thousands of URLs per hour through automation, it becomes a machine on machine war.Natalia Godyla:And on top of the speed and scale, it seems like there's also sophistication in the level of deception. You noted earlier that now it looks like a common user, they can spoof it. So can you talk a little bit more about that? So how does the ability to bypass our detection feed into them being more sophisticated?Donal Keating:Well, let me give you an example. The weakest link now certainly, in security systems, are the humans. So one of the things that most security systems are very good at is recognizing malware, when it can see the malware itself. So for instance, you have a macro embedded in a document, basically that can be detected relatively simply. Well, if you then encrypt that document and send it through an email, the mechanics of detecting the malicious payload is hampered by the fact that that document is encrypted. But then what you need to do is you need to socially engineer the person receiving the document to enter a password and deploy the malicious payload. And that's where I'm saying, people log in rather than hacking anymore. They can assemble enough information about somebody to make an email coming, even from an unrecognized sender to be sort of believable and to encourage a conversation.Donal Keating:And it's not a single email. If you're being targeted, like if you're a CFO or an admin of a system or something, they can be quite persistent over time. They can develop a relationship with that person and then eventually bingo, the malicious payload gets delivered. And they can send that in two parts. They can send an email and say, "Here is the password for the document that I am going to send you." That then, the human reaction to that is, okay, now I am expecting a document from this person. The document comes in and you have the password, that's social engineering.Donal Keating:Now, there are lots and lots of lists of username passwords. And what they tell everyone is, do not share passwords across different systems, especially your private stuff and your work environment. Well, if you're like me and you have a terrible memory, one password is a really attractive proposition. And you may not go with just one password, you make it really clever and add a one, two, three, four at the end of the password. But for people who are looking at thousands and thousands of passwords and millions of passwords, because they've been leaked, they can understand the patterns that people use.Donal Keating:The example is, if I'm trying to hack someone in Microsoft, I'm going to put the word Seahawks somewhere in the dictionary attack, because apparently that's what humans do. It's like, there are certain keywords that people trigger off and think, Oh, nobody can think of Seahawks. And I'm in Seattle. So let's say one individual is compromised in the company, that allows them then to log in to that account and then watch traffic. So what will they do? Someone might change their password, they don't want to be sitting on the email all the time. So what they will do is, they're going to your email preferences and they will forward emails that contain particular words. I've seen an attack where anything that has the word payment, invoice or bank in the email to forward it out to an external Gmail account. Then I don't need to get back into that account anymore because all of the emails containing those keywords are now being sent to me out on a disposable Gmail. I get to see all that email traffic.Donal Keating:So now I have one half of a conversation. And this is where the sophistication becomes really important. Somebody sends in an invoice, we'll say for payment. Well, when that invoice for payment comes in, now, someone has a template of an email that contains an invoice and all of the language. I take the person, the email who sent that invoice in, and I generate a homoglyph of it, meaning a domain that looks almost identical to the sender. Very often it can be even just a different TLD. So instead of, it could be Microsoft dot GZ. And I can use exactly the same username.Donal Keating:So now what I do is I insert a new mail into the chain, so I have the previous thread because they've been harvesting email from that person. And I now put it in my new email and says, whoops, there's a correction on the previous invoice, please change the banking information to this email. And we've seen this in phishing attacks. That sort of thing can be very pernicious. And that is quite widespread. That behavior of monitoring the email, the registration of a homoglyph, and then the conversion of a payment to a different bank account. We see that quite a bit now.Natalia Godyla:So how are we thinking about response to these new threats? What's next for security to combat them?Donal Keating:Well, all the time in the background machine learning, AI is getting smarter and smarter and smarter to protect the assets. And that's why in a lot of the cases that I talked about the objective is to get the username password, to commit that crime that is to login, not to hack in. Now, once they log in, they can do a lot of things. They will deploy remote access tools onto the network to enable them to do a lot of other things like the deployment of ransomware for instance. You need access to the system to encrypt everything. But that first step nearly always is the human element, the engineering, the human element to crack it open.Donal Keating:And, it's a bit like with COVID-19, we're told to wear a face mask, wash our hands and keep six feet apart. The things that we tell people to do are not new or exciting. Make sure you're using multifactor authentication, keep unique passwords for each site, make backups. All of those things, it's good hygiene. But for instance, the use of multi-factor authentication, I've not verified it myself, but I've seen statistics that say that in excess of 90% of username password compromises would have been thwarted if people had been using multi-factor authentication. So-Nic Fillingham:Some of the User ID team will quote 99% or greater. It's pretty significant.Donal Keating:Yeah. so that to me is the wearing a face mask and washing your hands of protection from cyber crime. I have a small carve out for nation state. If the Russians or the North Koreans want to go after you as an individual, you need to tiptoe very carefully. There's all sorts of nastiness that can be done to you as an individual. But the reality is for most targets, it is this people access, username password combination, they log in and then they start the progressive taking over the account to do whatever it is they do. The worst being ransomware.Donal Keating:It's not unusual. So, you talk about increasing sophistication. Ransomware was a big thing and then it took a hit. Why did it take a hit? Because people had deployed ransomware that were really destructive ware. They encrypted stuff and there was no keys existing. So suddenly everyone says, "Well, there's no point in paying a ransom because I'm not going to get my stuff back." So then the criminals had to go and do something else to prove that no, no, no. Really we can decrypt your stuff. So it's a kind of a marketing campaign.Natalia Godyla:There's something very comical about the fact that the hackers had to get people to trust them that they were going to do what they say they're going to do.Donal Keating:Oh, absolutely. Yeah, yeah, this is business no different from any other business. You get a bad reputation for something, you got to fix the reputation, or you got to get another way of leveraging people to do what you want them to do. And that's why I say there are people who are specialized in thinking up these social engineering things. They may not be coders at all, they may not know how to turn on a laptop, but they understand how humans work. There's other people then who are geniuses at writing the malicious payloads, writing the PowerShell scripts, obfuscating the PowerShell script so as normal detection won't pick them up that.Donal Keating:This is a whole stack of various things with various levels of sophistication and increasing sophistication. But the criminal will tend to go to the softest part of the ecosystem to make their money.Natalia Godyla:You mentioned that part of the challenge right now is that users are just getting smarter and so the hackers are responding in turn. If our users have been taught cybersecurity education on what is a phishing email, how is the evolution of education going to happen or what's next for education for the users so that they can prepare for this next wave of social engineering attacks?Donal Keating:A whole bunch of interesting things tumble out of that question. The first one is we used to always say, go look for mistakes in the phishing email. If it looks like bad English, it's probably phishing or whatever. I actually heard at a conference that they were sometimes deliberately put into an email to trigger the spiny senses of anyone who is halfway security savvy. And the reason was, the person who fell for the phish was then going to be more gullible. They were trying to cut down the amount of traffic that was coming to them for someone who would do... I'm talking specifically about something like tech support fraud, where you'd get an email that your computer was about to run out of its license key or it had some horrible vicious malware on it, and you needed to contact this number.Donal Keating:They would actually put in the sort of deliberate clues to anyone who was savvy. The result then, the people who were calling that number, were going to be much more gullible. So you also have to understand what is the goal of the criminal? And the phishing emails yes, they are getting much, much more sophisticated. But we especially in cloud, when you're looking at O 365 advanced threat protection, that description that I just told you, if something coming from and then another email comes in from microsoft.gz, we actually have exactly that detection running. These look alike domains, where you haven't communicated with that domain before, advanced threat protection will regard that as a high risk email.Donal Keating:So for-Nic Fillingham:That's a homoglyph, right Donal? You mentioned-Donal Keating:A homoglyph, that's exactly yeah, homoglyph. It means something that looks like another thing. So, the classic example are the Microsoft spelt and O, you replace it with a zero. The I you replace with a one. And this business has become during the election for instance, people will look for the registrations of all of the legitimate vote Arizona or whatever. it was I think, and of course, someone registered I think it was. It looks exactly like you would expect. The response to something like that for government especially, you should only be standing up state material on a dot gov domain.Donal Keating:So there are lots and lots of things that we need to educate people. Donal Keating:The IRS, for instance, will never ask you to pay your income tax with iTunes cards. You would wonder how does that scheme ever work?Nic Fillingham:Yeah.Donal Keating:But they say that Nic Fillingham:I've never had one of those phone calls, because I really want to hear the logic from the person that's trying to tell me what happened to the IRS as an institution where they now are relying on the consumer retail supply chain and the company Apple. And that's the only way they're able to accept funding. I want to hear that story straight from the person trying to try and pull the wool over my eyes.Donal Keating:One of the things we do actually is, we call these people. So, every time we get the numbers, there was one that-Nic Fillingham:I hope at home.Donal Keating:... we do actually what we call test phone calls. So, if you look at some of the other, I know this is not the subject of the podcast but, we've recently had big raids in India where 10 call centers were raided. All running tech-support scams. Taking people who thought they had something on their computer and paying subscriptions of to $300 a year to keep your computer protected. They are unsophisticated crimes. But the sophistication of persuading someone that they do have a problem is sophisticated.Nic Fillingham:Awesome. Well, Donal, thank you so much for your time. Again, the report that we're referencing here at the top of the conversation is the Microsoft Digital Defense Report. It's about 38 pages of fascinating insights into the state of cybersecurity. And a lot of the topics that Donal touched on in this conversation are elaborated on in much more detail there. We'll put the link in the show notes. Again, Donal, thank you so much for your time.Donal Keating:Very happy to be here. Thank you.Natalia Godyla:And now, let's meet an expert from the Microsoft security team to learn more about the diverse backgrounds and experiences of the humans creating AI and Tech, at Microsoft. Today we're joined by Michelle Lam, a threat hunter at Microsoft. Well, thank you for joining us, Michelle.Michelle Lam:It's a pleasure. Thanks for having me.Natalia Godyla:Yeah, of course. Well, can you start the show by just telling us a little bit about your day-to-day? What do you do at Microsoft? What is your day-to-day look like?Michelle Lam:Sure thing. So, I could tell you about the boring things, which is that, I look at a bunch of data and spreadsheets. And I look at them and I say, " Bad things happened, or everything is fine and people are off doing their normal things." But I guess the more complicated story to my work is that, what I look for is patterns in data that might indicate malicious activities. So that, might, could be anything from human-operated ransomware, to new malware strains, or even just new pivots in activity in general. So, things that we can feed into the rest of the Microsoft ecosystem for security.Natalia Godyla:And threat hunting is a relatively new space, correct?Michelle Lam:Yes it is. But I think it's interesting, because the concept of threat hunting has existed, but it's always been in other realms and security. So, if you think about things like security-tracking or security operation centers already looking at alerts and whatnot, or on the idea of incident response, the concept of threat hunting is already baked into a lot of these more traditional spheres of security. So, yes, it is new, but I think it's always existed in one form or another.Natalia Godyla:Do you feel like it's become a stand-alone part of security now? So it's been baked into these different aspects of security in the past, but now we need it as a stand-alone function?Michelle Lam:I think that really depends on where you're at? What kind of organization you're in? And what are you trying to do with that data? Because, it doesn't make sense to go hunting for data and the deep, deep sea of data that exists. If you have data that you need to analyze for a purpose, I think that's what threat hunting is really great for. For me, I'm looking for data because I want to figure out," what context can I give it that will be helpful to a customer? Or to the rest of Microsoft as a whole?" I think if you ask that question to anyone else in any other organization, then it's a different story because what part of that data is interesting to you is different for everybody, depending on your sector, depending on your organization, depending who you are even.Nic Fillingham:And what is that sort of, Stat focus area for you, Michelle. How do you scope down that near limitless sea of data for looking for threats?Michelle Lam:That's a fantastic question. I think I'm really interested in looking at different techniques that already are well known in the industry. So, things like using PowerShell, using Scripts, different ways of disabling security mechanisms. Those are techniques that already exist and can be used in one-off occasions. But what I'm really interested in, and when I look for this data is how I can correlate all of these little things that might happen one at a time, in a benign case. But if they happen all together, how can I combine that and say, "Is this related to a specific activity group? Or is this someone who's doing a penetration test? What sort of things can I identify about how they were executed or how they're launched? And can I make that connection to something else and provide that context elsewhere?"Nic Fillingham:Would you mind telling us about your journey into security? And then how you found yourself working for Microsoft?Michelle Lam:Sure thing. So I guess my story, even entering security really has to start with this journey of me entering tech as a whole. So, I myself, I come from a low-income family, and a family of immigrants. And so it was really interesting for me to decide what my career path was going to be as I started this journey of, "okay, well, I'm leaving high school. Where do I go?" And the direction that I was going to take was in the business direction. And I ended up deciding, with the encouragement of a few of the teachers that I'd had at the time to go into computer science. I won't lie, I was a little motivated by money, who isn't? But when I actually got into college and I discovered what you could really do in the field, I was really intrigued.Michelle Lam:And I tried to figure out, " What does it take to be more technical? And what else is out there?" So, while I was at college, I actually joined a security club. And there were a couple of students there that helped mentor me for the process of writing my own code, to do the very simple things like encrypting or decrypting data. And that moved on into me actually getting internships and learning how to code and ending up at Apple and working in cryptography and wondering, "what the heck am I doing? This is so cool, but I have no idea what I'm doing." So, my entry into cybersecurity was really fueled by this curiosity of, "I have no idea what I'm doing, but I'm going to continue to do it". And for me, that continued up until my last year of college. When for a lot of low-income and first-generation college students, there's this very common pattern of, it takes you a little bit longer to graduate from college because no one you've ever known has been through this process.Michelle Lam:And for me I was, to be frank, I was scared. I didn't know what it would mean for me to go out into the industry. So, I wanted to figure out what I wanted to do. And I wanted to figure out what to do in security. So, I actually attended a Women in CyberSecurity Conference, and I attended a talk by these two women that I really admire in the industry, Malware Unicorn and Maddie Stone. And they were super friendly and they did this course on Reverse Engineering and Assembly. And I was like, "Oh my Gosh. This is so cool. This is a field where I don't necessarily have to be coding, but I can put a lot of that low-level knowledge to use that I've learned in college and I can figure out what malware does. I can solve a problem."Michelle Lam:So I really took that into consideration as I moved forward. And I ended up teaching a course for my senior project about reverse engineering. I didn't know very much at the time, but that is what I decided to teach. And I also took an internship that was based in Incident Response and Computer Forensics at a government laboratory. And it was a super weird internship to have. It's not normal, I think for a lot of my peers to have that experience of, you go to a government lab, and it's a very different experience than what you expect. And you also reverse malware and you figure out what the baddies do. So, it's a little hard to explain to your peers, but I absolutely loved it. And I figured out, "This is what I want to do when I grow up. When I exit college and I graduate this is going to be it."Michelle Lam:So, that's my short story of how I got into security. And from there, it was a bit of a pivot before I ended up at Microsoft itself. So, after college, I had decided to go down this route of, " I can do a little bit of incident response. Okay, I'm going to take a job in incident response." So, I moved to Atlanta to take our role in incident response consulting, where I learned a lot. And they did a bunch of little things, but I didn't really know if I was advancing myself or learning about the baddies in the way that I wanted to. And it so happened that I attended a conference that's very focused on reverse engineering called REcon, which is in Montreal. And I met a few people that I'd actually met at some other security conferences when I was a little more junior in my college career.Michelle Lam:And I was like, "Well, what's going on?" And they're like, "Hey, I'm at Microsoft. I do cool things. You should come here and do cool security things too." And I was like, "But, are you sure?" And they're like, "yeah just chat, it'll be fine." Long story short, a few months later, I took a job offer from Microsoft, for my current team, The Microsoft for Experts team. And here I am getting to hunt on and look at really interesting data. So for me, it's been this really interesting journey of exploring and running into this field, and trying to figure out, how do you enter it without a ton of mentorship from those around you?Nic Fillingham:If someone listening to the podcast sees a bit of themselves in your story here, what would you recommend for how they maybe, go and find some of those support groups, maybe some of those mentors, maybe some of those industry bodies that could help them out early on in career, to get some of these experiences? Is there any tips or tricks you'd want to pass on?Michelle Lam:Yeah. So, I would say the biggest things for me were building a really strong network over social media. So that doesn't mean, go out and tweet all the time because, I certainly don't, but I definitely found a lot of really resourceful things on Facebook groups and Twitter groups. Even some of the internships that I actually applied to and got offers from, were things that were shared on a Facebook group for like Women in Security or Women in CyberSecurity. I only found out about a lot of conference sponsorships for following different Twitter feeds and seeing, "If I follow a bunch of these people, someone at some point is going to share some way that I can attend DEF CON or another conference for free or for reduced rate or some form of sponsorship. So, that's been really important for me as I grew my career and I definitely plan on giving back at some point because, I would not be here if it weren't for that.Natalia Godyla:It's interesting because I think for many of the people that we've chatted with, it's been a little bit more of a winding journey to security. But in your case you started with Comp Sci but you ended up thinking about security already when you were in school. So, how was that experience seem different than some of your other colleagues who have started in other backgrounds and have then made their way to security? Do you feel like it's been helpful to know that security was your path when you were in college? How does Comp Sci factor into it?Michelle Lam:For sure. So, in a way I do feel like it's been really helpful for me to join security and find out about security so early on, because, I feel like I've been able to learn a lot and be able to put a lot more of, I guess, some of the foundational computer science skills into use. Things like learning assembly which in college, if you're a college student right now and you're taking any assembly course, you're like, "I'm never going to write in this super low-level language. Why am I doing this?" Well, it so happens that when you work in this industry, you want it. Or if you take compilers. Compilers is surprisingly useful in security.Michelle Lam:So, I guess, what I think about a lot in terms of my career progression in comparison to some of my peers, is that I do feel a bit of a disadvantage sometimes because, I'm still quite junior in my career. I'm maybe two or three years out of college at this point, so there's still plenty that I have to learn, but I do feel that I don't have that traditional security experience. A lot of folks on Twitter and in the traditional security spheres, talk about this concept of, "You need CIS Admin experience to be a security person. You need to know all of these things. You need to have worked for 10 years, 15 years in security before you can become a threat hunter." And I'm like, "Did I make a career mistake?" To be honest, I have imposter syndrome about it quite a lot. But, if you think about it, everyone has this different take on what they're looking for when they're threat hunting.Michelle Lam:And what's valuable for me, coming from such a junior and such an almost indoctrinated security experience, is that I see these things and I see that they look bad, but I have a different way of relating to the data in which I might say instantly, "This is bad, and here's why," or "This looks weird," and someone's like, "No, you're wrong." And I'm like, "Well, you're just saying that because it looks like something you've used before. But I've never seen it and it looks malicious."Michelle Lam:So I think it's all about, there is a joy and a need for us to have different perspectives when we're hunting across data, and when we're looking across data. Because everything looks different to everyone, especially in this industry. And it's about, how do you take those arguments and how you condense it down to, "It's not argument. It's us trying to understand the data," that's really important.Natalia Godyla:So Michelle, how does AI and ML factor into your role? How do you leverage those tool sets to help our customers?Michelle Lam:We actually use AI and ML in several different detections that we use. Whether that be ranging from the antivirus in the AV side of things, to things like Windows Defender for endpoint. We might be looking at different signals and putting those together in different ways to figure out, if users are performing this type of recon several times in a row, that's malicious, that looks like exploration activity, right? There are other ways that we're looking at using it that might involve... We see this particular activity group perform this activity in sequence. When we see that, that's an indication to us that there is maybe this activity group is on this machine.Michelle Lam:And that's really interesting data for us to have, especially as we hunt and we track that data because maybe we're not completely sure the history of what we've looked at in security, I think, has always been very indicator of compromised base. It's been very focused on, we see these hashes, we see these files, we see these IP addresses, but what happens in a world when you can't really use that information anymore to hunt? For me, I'm really interested in when I see this behavior, how can I use that? I think that's something where AI and ML is super powerful and super helpful for us as we figure out like, if I were to move away from a world of IOCs, this is where we would go and this is how we would build a detection in order to actually catch a group in action.Nic Fillingham:We've already spoken to a few folks on the podcast, Michelle, that are working on behavior based detections and try and leverage ML and AI to do that. I'd love your perspective on your role as a threat hunter and what makes threat hunting as a process, and as a task, and as a role, what makes that sort of a uniquely human-based function, as opposed to simply a bunch of algorithms out there running in the cloud?Michelle Lam:I think there's two different ways to think about this. And one of them is that, well, how did the algorithms get created? You still have to teach the algorithms how to use that data. We are working with several data scientists to actually figure out how do we feed your algorithm that data that actually says that this is tied to an actor. And you can't do that without actually having a human to hunt across that data and understand what it means.Michelle Lam:I think the second component to that question is that attackers are human too. If they weren't human, then it would probably be a lot easier for us to catch them, and maybe we wouldn't be having this conversation, and maybe I wouldn't be having this job. But because attackers are human, we have to pivot ourselves to align with them. You can't expect machines to catch everything that a human is doing, but if we have humans that are looking at other humans activity, we might be able to predict and start learning off of what they're doing and build that into our algorithms so that algorithms can assist us to do the heavy lifting while we look for the new things that are happening.Nic Fillingham:I love it. That was a great answer.Natalia Godyla:This is a bit of a big picture question, but it sounds like a lot of your path to security has really brought you to this role to threat hunting. What would be next for you? Are you interested in continuing to pursue a career in threat hunting, or are you looking to explore other aspects of security down the line?Michelle Lam:I think that's a really wonderful question and it's tough for me to answer, being so early on. I think about a lot of the questions that you get asked about when you're pretty junior in your career, right? I won't lie, everyone has asked me, "What's your dream in five years? What do you want to do in five years?" And I'm like...Natalia Godyla:Every time you come home for the holidays.Michelle Lam:Yes. So, I don't know. I think about this a lot and I have to say, I actually do think I'm in my dream position right now. It's a different question of where I want to take my role and what I want to do with it really, because I love hunting across data. I love finding weird things. Like, what does this do? And how can I learn what it's doing? Nic Fillingham:What's the weirdest thing you found? What was your, like, you woke up in the middle of the night with like, "Oh my God, that was so weird?" Has anything stuck out?Michelle Lam:I want to say that I could answer that, but I'm not sure that I can actually share it, so it will just have to be a mystery.Nic Fillingham:Can you hint at something that doesn't jeopardize any OPSEC?Michelle Lam:No, that's kind of the joy about being a threat hunter. I don't want to share too much, I don't want to tip anybody off.Natalia Godyla:What big problems are you passionate about solving in cybersecurity? Are there any challenges that you're seeing that you'd like to tackle throughout your career?Michelle Lam:That's such a hard question to answer, because I feel like I am tackling a lot of really big problems as it is, fighting the fight against human operator ransomware is huge. But I think if there's anything that really is important to me in the way that I was raised and how I got into this career, it's about how do we make security and option for those who security might not have occurred as a first option? How do you make sure that security shows up for those that are underrepresented communities?Michelle Lam:Because it's not just a matter of physical security, but cyber security is so incredibly important for these communities. How can you make sure that they have access to it when they need it? There are a lot of scenarios that these communities have to reach out and figure out how they can get support in tough times in these kinds of situations. I would love to figure out what does that look like for me and for others.Natalia Godyla:I feel like this comes back to what you said earlier about all the communities that you can reach out to. It's always an aspect of you reaching out to try and find these communities. I think that proves out that some of these resources or niche are difficult to find right now, and that you have to put the effort into doing it. So just easing that access.Michelle Lam:For sure. And I think that's something that I've always struggled with, is this idea of how do I balance my career, progressing in my career versus helping the communities that I've come from. I've done work in the past, a volunteer with organizations like Girls Who Code, and we've brainstormed quite a bit internally of how do we volunteer our efforts to actually teach underrepresented communities, people of color, women who are younger, who might not traditionally come from a tech career path? How do we teach them these cybersecurity skills? Because we're constantly running out of cybersecurity professionals and the only way to solve it is to grow the base of cybersecurity professionals that exist. So how do we teach them and how do we introduce them to this field in a way that makes them feel like they belong?Michelle Lam:I feel like that's a really important problem to solve, especially because I come from a place where had I not gotten lucky at college and ran into a club full of cybersecurity people, maybe I wouldn't be here. And for me, that's scary to imagine because I love what I do. And I love that I get to feel like I'm saving the world. So what does it mean if I teach others to do that? How do I do that? That execution is... I don't know, the idea of that is so interesting to me and I think there's a lot of impact that I could have.Nic Fillingham:Michelle, are there any organizations you want to plug?Michelle Lam:I would like to talk a bit about Blackhoodie, which is this really awesome organization that was founded by a couple of ladies off of the Twitter security community. It's really a community of women who are teaching these reverse engineering workshops that are meant to be technical, and to really teach you about technical low-level skills that could get you into reverse engineering or into the security community. All of the women that I've met from being a part of Blackhoodie have been absolutely amazing. I stay connected to them to this day, and I've even taught a course for them at a previous Microsoft conference, BlueHat. If you are a lady listening to this, I would super recommend that you go check them out on Twitter and see if they've got any courses coming up that you might be able to attend because they're free and they're taught by some really, really intelligent woman across the security industry.Nic Fillingham:What do you like to do in your free time, michelle?Michelle Lam:That's a really great question. My favorite-Nic Fillingham:Apart from quarantine for eight months.Michelle Lam:Okay, fair. Quarantining is a fantastic hobby. My hobbies are drinking lots of bubbly water, playing with my puppies, and fashion. I love fashion. Someday, if I'm good enough, I would love to compete with Jessica Payne in Malware Unicorn. We'll see if I get there, but I want to have a security idle fashion competition.Nic Fillingham:As in where you make clothes? No, what would that look like?Michelle Lam:I don't know. I guess we could all just attend a security conference and wear ball gowns and I don't know, compete against each other. I'm not sure what it would look like.Nic Fillingham:Tell us about your puppies.Michelle Lam:Yes, I have two puppies. One of which was obtained during coronavirus, her name is Kali, after Kali Linux. Very secure. And our other pup is Nelly, who is a beautiful rescue.Nic Fillingham:Do they have an Instagram account?Michelle Lam:No. I mean, even if they did, I'd like to maintain a little bit of OPSEC, so maybe not. Sorry.Nic Fillingham:Well, Michelle, we're very happy that you found your path to both security Microsoft and thank you for doing the work that you do and best of luck helping others find their path as well.Michelle Lam:Thank you.Natalia Godyla:Well, we had a great time unlocking insights into security from research to artificial intelligence. Keep an eye out for our next episode.Nic Fillingham:And don't forget to tweet us @msftsecurity or email us at with topics you'd like to hear on a future episode. Until then, stay safe.Natalia Godyla:Stay secure. See for privacy and opt-out information.
In this episode, hosts Nic Fillingham and Natalia Godyla speak with Sharon Xia, a principal program manager for cloud and AI at Microsoft, about the role machine learning plays in security. They discuss four major themes, outlined in the Microsoft Digital Defense Report, including how to prepare your industry for attacks on machine learning systems, preventing attack fatigue, democratizing machine learning and leveraging anomaly detection for post-breach detection. Then they speak to Emily Hacker, a threat intelligence analyst at Microsoft, about her path from professional writing to helping find and stop attacks. In This Episode, You Will Learn: • How to prepare for attacks on machine learning systems • The dangers of a model poisoning attack • Why it’s important to democratize machine learning • How a humanities background helps when tracking threats • The latest methods attackers are using for social engineering Some Questions We Ask: • Why are most organizations not prepared for ML attacks? • How do you assess the trustworthiness of an ML system? • How can machine learning reduce alert fatigue? • What kind of patterns are analysts seeing in email threats? • Why is business email compromise treated differently than other threats?  Resources Microsoft Digital Defense Report, September 2020’s LinkedIn:’s LinkedIn:’s LinkedIn: Natalia’s LinkedIn: Security Blog: Transcript(Full transcript can be found at Fillingham:Hello, and welcome to Security Unlocked, a new podcast from Microsoft, where we unlock insights from the latest in news and research from across Microsoft Security Engineering and Operations teams. I'm Nic Fillingham.Natalia Godyla:And I'm Natalia Godyla. In each episode, we'll discuss the latest stories from Microsoft Security, deep dive into the newest threat intel, research and data science.Nic Fillingham:And profile some of the fascinating people working on artificial intelligence in Microsoft Security. If you enjoy the podcast, have a request for a topic you'd like covered, or have some feedback on how we can make the podcast better...Natalia Godyla:Please contact us at or via Microsoft Security on Twitter. We'd love to hear from you.Nic Fillingham:Hello, Natalia. Welcome to another episode of Security Unlocked.Natalia Godyla:Hello, Nick. How's it going?Nic Fillingham:It's going really well. We've got some early data, some early data hot off the presses from our listeners. I thought we might jump straight into that, instead of finding out what smells have permeated my basement. Is that-Natalia Godyla:Great to hear it.Nic Fillingham:Yeah. So, we just got some data coming out of the various podcast hosting platforms, and we have been listened to in over 60 countries, which is, I mean, that's amazing. That's if my math is correct, that's a quarter of all sovereign nations on earth. So that's pretty cool. Right?Natalia Godyla:Yeah, we're making headway. I feel like global just makes it sound like such a big deal. We're currently listened to in Estonia, Kazakhstan, the UK, both of our father slash motherlands Australia and Poland. So, it's great to see the representation. Thank you all.Nic Fillingham:I want to list a few more, because I just want to make sure that the few listeners that I think are there, they're getting a shout out. Myanmar, Azerbaijan, Albania, Haiti. Thank you so much to all of you listening to the podcast. On today's episode, we speak first with Sharon Xia, who is the Principal PM in the Cloud Security team. This will be the first of five or six interviews we're going to have over the next few episodes with authors and contributors to the Microsoft Digital Defense Report, the MDDR. You can download that at This is what I like to call the spiritual successor or the successor to the Security Intelligence Report, the SIR, which is a document that Microsoft has produced for the last 15 years on trends and insights in the security space. Natalia you've read the report. What would you say to folks that are sort of thinking of downloading it and giving it a read?Natalia Godyla:Well, first off the machine learning attack section is definitely one to read. It's fascinating to read about the new attacks that there are, model poisoning, model inversion, we'll touch on them in future episodes. So I'll leave it at that, but lots of new goodness, and just in general, the MDDR is a huge effort within Microsoft. It's highly collaborative and it brings together a ton of experts who really know their stuff. And so you'll see just that breadth of knowledge and intelligence when reading the report and in all of our upcoming episodes, since we'll be spotlighting, a number of experts who were contributing to the report, we also, in addition to the MDDR, we'll have Emily Hacker on the episode who is a threat analyst, and she'll talk about her journey from literature major to cyber security realm.Nic Fillingham:Awesome. We hope you enjoy the episode, Sharon Xia. Thank you so much for joining us. Welcome to the security unlocked podcast.Sharon Xia:Hey everybody, thank you for inviting me.Nic Fillingham:Oh, you're very welcome. We're happy to have you, could you give us sort of a brief introduction to yourself? What's your title? Tell us about what you do. Day-to-day in your team, sort of the, the mission and goal of your role and the team that you run.Sharon Xia:Sure. So I'm the principal program manager, which manages the PM team in Azure security data science team. And we have six PMs with 30 data scientists. Our day to day work is using machine learning to write threat detections and other features that protecting Azure, protecting our customers and also protecting machine learning models.Nic Fillingham:So that's a team of 30 data scientists, sort of machine learning experts, that are protecting all of Azure and Azure customers. Is that right?Sharon Xia:That's right. So actually including more than Azure customers, because our products and our solutions applies to on-prem system, as well as, as a crowds like AWS and the GCP.Natalia Godyla:Microsoft had recently published the Microsoft digital defense report, in which we talked about machine learning and security. And as I understand that you contributed to this report, and one of the themes was something you just touched on, which was preparing your industry for attacks on machine learning systems. So can you talk a little bit about how the cybersecurity space is viewing these machine learning attacks? What's happening? What are the measures organizations can take to protect themselves against these attacks?Sharon Xia:Yeah, as we all know, machine learning takes an increasingly important role in the operations and in our day to day life, right? It applies to not only like a facial recognition or voice, or even apply in many medical devices or analysis.Sharon Xia:So it's just embedded in our day-to-day life nowadays. But to the attacks, cyber attacks to the machine learning system and the machine learning models, we're just getting to know these. And it's more and more prevalent, based on our research. We did a survey to 28 large customers, enterprises, 25 told us they have no idea what are the attacks. You know, it's there. And to the machine learning system. So that's kind of alarming, right? And for example, the model poisoning attack, and real world example is, attack can manipulate the training data to make a street sign classifier, that to learn, to recognize a stop sign as a speed limit. So that's really dangerous if you think about it, right. If you're driving a Tesla and you're supposed to stop. I'm not saying Tesla is vulnerable to this attack, but this is kind of an example of a model poisoning attack.Nic Fillingham:So, we talked about the report. So the digital defense report, the Microsoft digital defense report that was released, it's a pretty lengthy document. It's full of a lot of incredible guidance. You and your team specifically contributed. And what we're talking about on the podcast today to the section within the state of cyber crime, which is called machine learning and security. And as you, as you just touched on that, the very first of the four trends that are called out there is simply just awareness, and preparing. I want to just touch on that stat that you mentioned just a minute ago. So you surveyed 28 organizations, 25 of those 28 just said that they don't have a plan for, they don't have tools. They're not prepared for adversarial or ML. Is that an accurate takeaway?Sharon Xia:Yeah. So what do we, we seen at this moment is a security team and the machine learning team are running on two parallel orbits right now. So they know to not interact, that they are doing their own things, not aware of security on machine learning system. Yeah. So the first step we, we have been putting a lot effort is the community awareness. And we definitely need community help to pull those orbits together. Finally, interact, right? So that's a call to the community. Like that's a raised that awareness and walk together to first aware of these, then due to some tools, trainings to get our defense up, you have red team and a blue team, right? So they'll get our defense up to the speed.Nic Fillingham:You mentioned a few types of sort of attacks there against models, model stealing, I think is relatively self-explanatory. Model inversion is interesting the way you explained it, it sounds like it's the ability to sort of reverse engineer or extract the data out of a model. The one that I sort of want to touch on here is, is sort of model poisoning. So you, you explained it as poisoning a model so that instead of seeing a stop sign, if it was trying to identify road and traffic signs, it may see something else. It may see a speed limit or something. How does that happen? How do we know how model poisoning works? Have we seen it in action? Have we been able to sort of post-mortem any successful model poisonings to understand how it actually happens?Sharon Xia:Yeah. There are multiple ways to have the model poisoning happening because the- like I described, it's about manipulating the training data, right? So if you have access to the training data directly, you could manipulate it, but that- on purpose that needs some machine learning knowledge to do it right? So you can also, let's say if at a first glance, you don't really have the access to the poisoning data, but then you have access to the network. So you can do a traditional main, the middle attack, to disrupt the training. And there are two kinds, integrity attack or availability attack. So if you disrupted the training model to run the training effectively, this is basically kind of attack from availability point of view. And if you change the data, like the street sign classifier, to make it read to us a speed limit, that's called a kind of integrity attack.Sharon Xia:So there is some multiple ways to do that.Natalia Godyla:So how are we thinking about assessing the trustworthiness of an ML system? It sounds like it's clear that we're still at the awareness stage and we're partnering with organizations to build out frameworks. What elements are we bringing into these frameworks or standardizations to measure trustworthiness of ML systems and identify whether they've been impacted?Sharon Xia:Yeah. We came up with kind of an amendment to our item, Microsoft, an amendment to our security development to ripe cycle. One of the process is the threat modeling. So we have machine learning threat detection, the threat modeling for machine learning systems. That's at a specific guidelines, questions, how do you do threat modeling on a machine learning system to identify, those potential attack surfaces and the potential risks in the development process? So that's the first step we are taking to, this is also part of a awareness effort, right? When you are doing the regular threat modeling, and you are asked for these questions, for example, if your data is poisoned or tampered with how would you know? Right? So then the follow-up question is, do you have telemetry to detect a skewed data in quality in your training data? Right. And are your training from user supplied inputs? Sharon Xia:If Yes, right. What kind of input validation or sanitization are you doing or if your training is against an online data store. So what steps do you take to ensure the security of those connections? There are long list of questions we ask in our, regular threat modeling like that. We actually published the document a while Microsoft security engineering site, it's a public documentation, with all these questions for the community to reference.Nic Fillingham:Sharon, what should, Microsoft customers know about how we are securing our AI systems and machine learning models that are in production. Obviously we're doing everything we can, we're investing heavily, but this is a very new area.Sharon Xia:Right. Yeah. So like I said, at the very beginning, we work with Microsoft scale, right. The Scott Battery register, they all aware of the effort. So we will work with the responsible AI at the Microsoft white. Also, we have an ISA working group that focus on, responsible AI and adversarial AI. So it's a Microsoft's effort to make sure at our engineering part, we are building a secure machine learning system.Natalia Godyla:And aside from protecting our machine learning systems, how are we taking this technology, taking machine learning and applying it to our security solutions so that we can empower security teams?.Sharon Xia:Good question, we're building solutions, detections in our cloud native, SIEM product, Azure Sentinel. So it's not being released yet, but we are working on it so that, our customers can use the tech knowledge based on our experience, our study and to apply it to their machine learning systems, to at least, detect those attacks to their machine learning system. And another end is we have red team actively, doing red teaming activity to the machine learning system. And we also keep learning the new attack techniques in that way.Nic Fillingham:Got it. So we've covered that first trend here, which is really about awareness of this new category, of this new sort of threat of attacks on machine learning systems. I might move on to that the second of the four trends that are in the report and that one is talking about leveraging machine learning to reduce alert fatigue. Can you talk a bit about that trend for us, what happened in 2020, or sort of in the last sort of 12 months around how ML has advanced in the use of ML to help reduce alert fatigue?Sharon Xia:Yeah. So, when you look at the security operations, the security analysts in every organization are dealing with a lot fatigues. I think if you are working in security operation field, you have to deal with salient alerts from different products like Enterovirus or Pareto Network, firewalls, and then EDR solutions, XDR solutions while for, all these kinds of security solutions, just sending alerts like a thousand alerts. So a typical, security analyst in the security operation center for an S 500 enterprises, they get about, 2000 alerts. They have to deal with daily that's obviously cause lots of issues, right? So on the other end, if you're not able to go through all these alerts and you may drop off the real attacks, but all these alerts, there are lots of false positives. So there is a survey saying some products generate more than 50% false positives, or even 70% false positives that really preventing the defender team, the SOC analysts, to deal with the two attacks, real threats.Sharon Xia:So one of the reason why are all these false positive is because the tradition or low based approach doesn't adapt to the change of the environment. The advantage of machine learning is it learns that new environment, right. And adapts to the change of the environment. And so we are looking at the Azure Sentinel, we have this machine on threat detections called a fusion. Fusion Technology use three different machine learning algorithms and a power, provide a graph and use kill chain and use different machine learning algorithm. We basically correlating signals from multiple products, multiple sources like your identity management system, your firewall, your EDR, your end points, also sources of data and they lock, all these anomalies and chain them in together in the sense of the kill chain, threats and the kill chain sense and fired like a high fidelity alerts.Sharon Xia:So give you an example. If you find a suspicious login from a tall Barraza meaning, an anomalous IP address, then this is maybe not that suspicious. But it's not meant a high fidelity, like this account is compromised or this login is malicious, right. But then if you follow by unusual mass download or setting up a mailbox, forwarding law in outlook and the forward, all the company, business email to a Gmail or something like that, those activities, if you chain those activity together, you can see obviously there is something like a data ex filtration or seek to attack, depending on different signals, right? So this is how we use machine learning to alert, reduce alert, fatigue and give you high confidence and high fidelity alerts. Allow the security analysts to focus on, these, their energy to investigate and mitigate those threats.Natalia Godyla:The volume of signals and the need for specialized skill sets, data science skills to develop these ML models. That brings us to a third theme, which is democratizing ML. So can you talk a little bit about, what our ask is to the security community and how we view democratizing ML as a next step in the progression.Sharon Xia:In a way we've seen in the industry, we're short of security experts. We are definitely short of, data scientists to build good, high quality threat detection. We need to boost knowledge. Security knowledge, as well as machine learning knowledge and going further. We also need domain knowledge, which I mean, industry domain knowledge is if it's a financial industry or healthcare or energy, or Microsoft, we have cyber security experts, right. For IT, information technology. We also have, hundreds of data scientists like my team, have 30 different full-time data scientists. So we also work like across the team, we work with our threat intelligence team, we work our security analysts team leverage their knowledge. So when you use the product we produce at a Microsoft like this threat detection, it's the result of multiple teams, multiple efforts, all the expertise in there, but we don't claim we know everything.Sharon Xia:And like I said, a generic machine learning, algorithm may work well in one environment, but less effective in another environment because of some special circumstances in that organization. And we fully realize, there is a lack of resource of data scientists in the enterprises. So what do we want to do is enable security analysts. Experts in security and their domain expert in their organization. To be able to improve the beauty in machine learning models, being our products, for example, Azure Sentinel to include quality of the model produce better signal in their environment. So this is the effort of democratizing machine learning in the SOC ML. So we are building this interface and this technology and in the product. So security analysts can customize our machine learning models without any machine learning.Nic Fillingham:And Sharon that leads us to sort of the fourth and final sort of big trend that's in the report. And again, this is the Microsoft digital defense report, 2020, which you can download at and Sharon, that sort of final trend that's discussed here is about leveraging anomaly detection for post-breach detection. We had Dr. Josh Neil on the podcast. I think in our second episode, his team is actively involved in this area. Can you talk a little bit about the sort of final trend that's called out in the report?Sharon Xia:Yeah. So behavior changes over time, right? And that's the beauty of machine learning. So, machine learning model, we observed the normal behavior. And then we signal if there's anomalous behavior happens, unusual activities, and these are important for the post-breach detection. If we observe anything abnormal happening, we stitch all these abnormal together and then find those strong attack relevant incidents. So there are the supervised machine learning models and the unsupervised machine learning models. And when we found out, because supervised machine learning models requires labelling and this put lots of demand on our customers. So we are actually now switch to more and supervise the message to attack, detect those behavior changes or abnormal behavior changes that will automatically adjust in a profile, a user or a machine or IP. We call those, all of them entities in the customer environment and they learn those normal behavior versus abnormal behavior. So that's how we, use anomalies to detect those post-breach detections. And because of these kinds of unsupervised machine learning models.Sharon Xia:Most of the models, we are able to do streaming fashion because it doesn't require training. So to be able to do streaming fashion, which is bring us to the meantime, to detect in the milliseconds, right? This is important. If you can detect a potential compromise in near real time, we want to do that, right. Otherwise like "Oh," nine months later, or maybe two days later, you'll find a compromise, right. So-Nic Fillingham:If it's not instantaneous, it's sort of useless.Sharon Xia:Right, I know, yeah. So this is really a truly important advantage in tech knowledge. We are able to detect those anomalies in real time or near real time and stitch them together as quickly as possible.Nic Fillingham:Well thank you, Sharon. There's a lot in the five pages of the machine learning and security section of the report, there is a lot of content to cover and we've really just touched on each of those four trends.Nic Fillingham:I highly encourage folks to download the report. We'll make sure the link is in the show notes. If you're someone that can hear links and remember them and put them into your browser, it's Xia:Yeah. What I wanted to say is it's very exciting that we are working on really this important area, and protecting our customers with machine learning technology, right? And there are lots of new areas, new territory we haven't explored. So I would really call for the community together to work with us and to innovate in this area, so our customers are better protected.Natalia Godyla:That's great. Yeah, it'll be a group effort. Well Sharon, thank you for joining us today. It's been great to hear about the progress we've made and the progress we are making in machine learning and security. So really appreciate you walking us through this and sharing the great work your team is doing.Sharon Xia:Thank you for the opportunity.Natalia Godyla:And now, let's meet an expert in the Microsoft security team to learn more about the diverse backgrounds and experiences of the humans creating AI and tech at Microsoft. Today, we're speaking with Emily Hacker. Thank you for being here, Emily.Emily Hacker:Thank you for having me.Natalia Godyla:Well, let's kick things off by just talking a little bit about your day job. So can you tell us your role at Microsoft and what your day-to-day looks like?Emily Hacker:Yeah, definitely. So I am a threat intelligence analyst on the TIGER team on Microsoft Defender. And I spend my days doing a variety of things. So specifically, I have a focus on email threats. So I gather a lot of information about email threats from open-source intelligence, from telemetry, from internal teams. And I combine all of these sources to try and find the email threats that are impacting our customers the most, and to put in proactive measures to stop those from impacting customers.Nic Fillingham:I want to know what the TIGER team is. What's a TIGER team?Emily Hacker:A TIGER team. It does stand for something, Threat Intelligence Global-Nic Fillingham:Is it a backronym? Were you all sitting in a room, and you're like "We need a cool name"?Emily Hacker:Oh, for sure. Definitely a backronym. It was definitely a backronym.Nic Fillingham:Someone's like "Tigers are cool"?Emily Hacker:Yeah, I feel very confident.Nic Fillingham:So you made it work.Emily Hacker:Yeah.Nic Fillingham:You made it work, but it's not necessarily memorable?Emily Hacker:No, we do have a lot of tiger imagery and logos and stuff related to our team now. And so we know what animal we are, but we might not know what we do.Natalia Godyla:I love that you guys went all in on it.Nic Fillingham:Are there any other teams based on animals of the Serengeti?Natalia Godyla:No, oh the Serengeti. So there's a fishing org that I've dotted a line to that we recently backronymed as well. And now it's Osprey, like the bird. So I'm like a member of the animal kingdom here.Nic Fillingham:Yeah, that's like a seagull, isn't it?Emily Hacker:I think they're pretty scary looking though. I think that was more the imagery.Nic Fillingham:It's also the name of the big Marine helicopter I think in the British Navy.Emily Hacker:The helicopter, yeah. And that's what I usually think of first. I think it's the one, the helicopter that maybe folds up or something.Nic Fillingham:That's got the wings that fold out? Is that right? It's sort of like half a plane?Emily Hacker:Yep. Mm-hmm (affirmative).Nic Fillingham:It's like a VTOL, is it a VTOL?Emily Hacker:It's fancy looking for sure.Nic Fillingham:Got it. Well, this has been a great conversation. Thanks, we're done here. No, I think you were... I'm sorry, I derailed us by asking what TIGER stood for.Natalia Godyla:I was going to start with a rather broad question, so I'm glad we did TIGER first. So you spend your day-to-day on email threats. Do you see any patterns that... like to elucidate the audience on?Emily Hacker:So patterns, I mean we see a lot of different techniques and patterns and stuff that we're tracking for sure. I think with... We look at both malware threats being delivered by email, and we look at phishing, like credential theft, threats being delivered by email. And one of the things that I would say, maybe a pattern that I've noticed is that a lot of times the techniques that we see between the two are kind of different. So it's usually noticeable to us if we're looking at certain techniques that is definitely malware versus fishing.Emily Hacker:And then we've also recently expanded more of our deep dive into business email compromise, which often is completely wholly different from the other two types of threats that I just mentioned.Natalia Godyla:Can you describe why business email compromise is often treated wholly different? What is the distinction between that and the other two threats?Emily Hacker:Yeah, definitely. So business email compromises a lot of times is totally different from malware and phishing, because it won't contain any links or attachments. So it's totally social engineering based, which is interesting to me. Personally, I find it super interesting because it's basically just the quote unquote "Bad guys" if you will, tricking people into wiring them money.Emily Hacker:So when we're looking at malware threats, a lot of times they're going to use links or attachments that lead to obviously malicious code being downloaded onto the machine. And the emails themselves might be... We've seen completely blank emails. We've seen emails that use really generic lure, such as "Please do the attached invoice." Of course, the attached invoice is fake. And with phishing, similar we'll see lure such as... Actually we see a lot of they're like "Please join this Zoom call or this teams call or whatever."Emily Hacker:They're going to try and make the recipient click on the link. But with business email compromise, it's totally done in email. So the threat actor will just send an email. A lot of times they will either compromise as the name suggests, they will compromise one of the accounts of a individual who works at a victim company in accounting or wire transfers or that kind of job. And they will send emails from that account. Or another thing I've seen is they will have some kind of methodology of watching emails on a victim's email network. So either via some o-off phishing that they had done earlier, or perhaps they got credentials to the email inbox. But then when it actually comes time to send the malicious email, rather than using the user's email, they'll create one that looks almost identical, but just change a couple of characters.Emily Hacker:So they might register a domain. For example, if someone was trying to use my email address instead of "", they might register "Micros0ft, with a", And then use my exact username. So to an unsuspecting victim, a reply to a thread will look exactly like it came from me, but then the malicious emails themselves aren't going to contain links or attachments. They're literally just going to be the bad guy saying like, "Hey, can you wire me these hundred thousand dollars or more, send it to this bank account?" And since there's already a level of trust with the victim, because it's usually coming either from a legitimate email account that they're used to doing business with, or one that's faked to look very similar to it, these are super successful.Emily Hacker:The people are wiring money to attacker accounts. And there's no malicious code involved. There's no phishing link involved, it's completely social engineering. Sorry, that was a really long answer. I got apparently really into that, sorry.Nic Fillingham:Emily, I wonder if you could tell us how you found your way to Microsoft. Have you been in security for a long time? What was path into your role and how did you find yourself in the security industry?Emily Hacker:Definitely. So it's definitely a bit of a roundabout interesting story. So it goes back a ways to when I first went to college, I guess. So I have a degree in English and communications and a minor in journalism. And I had every intention of being a newspaper reporter. I worked for my school's newspaper for a while. And then I worked for the city newspaper, for the city that I went to college in. And upon graduation, I decided maybe I wanted a job that had a little bit more normalcy. I really loved newspaper reporting, but it was a lot of late nights in the newsroom and stuff. So I ended up going into technical writing, and my first job out of college, I was actually writing software manuals. So it was pretty dry stuff, I'll admit. Where I was writing the manuals that people would refer to if they were having trouble.Emily Hacker:This was specifically for software for car dealerships, where the stuff I was writing was like "Press the F5 key to submit", or like that level of manuals, those very dry manuals. And I wasn't all that excited by that work. Some people love it and I understand why, but I didn't. So I was lucky that a girl that I had worked with at that job, I had only worked with her for a couple of months and she had gotten another job. Well, she contacted me about 10 months later and said that she had gotten promoted and wanted to hire me to backfill her. And she said it was a tech writing job, but it was totally different from the type of tech writing that we had been doing previously at the company. So I gave it a shot. I applied and I went to work with her.Emily Hacker:And what it was was I was actually the tech writer for a threat intelligence team at an oil and gas company, but it was my first foray into security. And it was not something I even knew was a thing honestly before, I didn't realize cybersecurity was kind of a field that people could work in. And it was very exciting to me. And I remember the first year or so that I worked there, everything was new and exciting, like "Oh my God, threat actors, what are those? This is so exciting. Nation States, Oh my God, this is a thing that's real." And it just all seemed like this movie script, except it was real. And after a bit of doing the editing and stuff for their reports, the reports that I was editing were very interesting to me. And I would ask questions because I needed to, to understand the report in order to edit it.Emily Hacker:But also just because I was legitimately interested, like "How did you do this analysis? What is this?" And I quickly decided I liked their job better than mine. So, I decided I was going to learn from my coworkers. And I am extremely lucky that the team of threat intelligence analysts that I was working with are some of the best people I've met in my life at that job and were super open to helping me learn. If I would say like "Hey, what are you working on? Can I kind of sit with you and learn from you?" Everyone was always just like "Yeah, let's do it, let me show you what I'm doing, blah, blah, blah." So I learned from them, and eventually, there was a time where we were a little short-staffed, as is common in security. And we were in charge of checking the phishing email inbox.Emily Hacker:So when users at the oil and gas client that I was working for would submit potentially suspicious emails, they would all go to an inbox that we had to analyze to determine if they were malicious or not. And it was a time-consuming job, and we just didn't have enough people on the team to do it and the rest of our work. So I kind of volunteered to help out. And that was how I got to learn how to do actual analysis. And I had job duties related to analysis. So I learned pretty much completely on the job from my coworkers. And then from there, I did that for about a year, maybe a little bit more after that. And I decided I wanted to move to Seattle, I was living in Texas during that.Emily Hacker:And I was very interested in living up here in the Pacific Northwest. So I left that job and got a job as a security researcher at a security vendor here in Seattle. So it gave me that other side of security that really allowed me to see the full picture of both having worked at a SOC, having worked at a vendor. And then I did that for just over a year. And this position at Microsoft opened up and I actually applied, Emily Hacker:I don't want to say as a joke, but I didn't think I was going to get the job.Nic Fillingham:As a stretch.Emily Hacker:Yes. It would be like if I applied to be president of the United States or something. It's one of those, where I'm like, "Oh, wouldn't that be great to submit the application," thinks never again about that moment. And then I was shocked to say the least when I got called for an interview and even more shocked when I got offered the job. So that was back in March. So I've only been here for a few months and I am loving it obviously so far. And what is really exciting to me is how this job is kind of, I get both the focus of having in-point telemetry like I did at my first job and phishing email telemetry. And then I also have a wider birth of just a lot of data and open source intelligence like I did it at my second job. And now I have them both here as well as getting to work with some of obviously the smartest people in the industry. So it was very exciting and I still am a bit amazed that I work here.Nic Fillingham:When you were writing manuals in for the car dealership and probably thinking about what was going to happen in the future, was there a little kernel, was there a little nugget of, it'd be awesome to be a company like Microsoft and doing cool nation state security, investigatory stuff?Emily Hacker:Absolutely not. I didn't even know that this was a job opportunity. The fact that this is a job that people do and now that I do. When I had first graduated and gotten my first job out of college, there was just so much about the world that I didn't know, but there was so much about careers that I didn't know. I didn't even know this was an option. And I do remember distinctly, I wasn't a huge fan of that job, but I didn't know what else was out there. And it just feels, everything's very overwhelming when you're 22 years old and you're like, "What is life like? Is this what I have to do forever?" So I'm just glad that I now know that this is an option.Nic Fillingham:What is life? Guess what? You keep asking that question. I'm afraid it's continually one you keep going back to. In a good way though. Do you find yourself bringing your technical writing skills, your formal sort of literature training? Do you find you're bringing that into this current role?Emily Hacker:Yes.Nic Fillingham:Are you writing a lot of reports and does that help you?Emily Hacker:Amazingly so much so that I think that this is something that people who work in technology don't always think about, but I work in threat intelligence and a large, extremely important facet of threat intelligence is communicating that intelligence to decision makers. If you know what's the intelligence but you're unable to communicate it, it's useless. So we write a lot of reports. I have a lot of those skills from my previous work. So writing a report is not difficult for me. It's something I've literally used to do for a living and knowing exactly how to phrase technical situations in a way that everybody, including non-technical people can understand is something I'm very good at because I have historically been a non-technical person. So it's something that is very useful to me.Emily Hacker:The other people who work on my team are also very good at it. But my point in that is that a lot of them have tech backgrounds. They have degrees or jobs where they have worked in technology. And so they have that tech skillset, but they have to learn the writing and communication on the job. And I have the writing communication and I had to learn the tech skill set on the job. And now all of us are good. We all do the job and we're all very good at it and we all have our things that we specialize in and we can help each other. But the point being when it comes to working in security or technology and hiring for security or technology, there's a large swath, if you will, of skillsets that are needed and nobody's going to have all of them for the most part. So finding people that have some of them, they can be trained up in the other ones, even if the ones that they're being trained up in are the technology ones.Nic Fillingham:Yeah. So have you found yourself in the same way that your colleagues were sort of helping you in the early days? Learn, fill in gaps, if you will, with you sort of being sort of somewhat new to the industry? Have the tables now turned? Are you now helping your colleagues be better communicators and helping them in their ability to pass this intelligence on into way that people understand?Emily Hacker:Yeah, I think so. So I definitely have edited a few of my colleagues reports before they went on to the formal editing process and just kind of taking the time to sit with them and be like, "This is what I'm changing and why." Either A, it's grammatically incorrect and let me explain to you what grammatically correct would be, or I'm saying this is unclear and we can make it more clear by saying this or this is too technical, only a handful of people reading this are going to know what this means and we need to simplify it to layman's terms. And I think people appreciate it. I hope. Either that or I'm like the red pen girl who just comes in and ruins everybody's reports and they're all terrified to see me coming. But I do think that they appreciate it.Nic Fillingham:What do you like to do Emily?Emily Hacker:Yeah, I do things.Nic Fillingham:Good answer.Emily Hacker:Okay. Believe it or not, I live in the Pacific Northwest, so I like hiking. I know. So does everybody in the entirety of the Pacific Northwest, but I actually really like hiking and that's why I moved here from Texas. So that's something that I greatly enjoy. I do things at home. Oh my God. I actually had made a list. This is sad. But at one time I made a list of things I do for fun, because when people ask this question, I always forget. I like writing. I did go to school to be a newspaper reporter. I still like writing. So it's my goal one day to get a novel published, but they may never come. And I play music. So I play several instruments and I like running. Do I like running? I run whether or not I like it. It's questionable.Nic Fillingham:Does anyone really like running?Emily Hacker:I don't think so.Natalia Godyla:I actually immediately want to ask what genre novel would you write?Emily Hacker:I think I would write a mystery, detective novel, because I'm really into true crime, which also everybody. But I like watching a lot of stuff about true crime, but then I'm also really... Am I admitting this? Probably. I'm also really into paranormal stuff and Big Foot and ghosts and what are they doing? And whether or not I believe in them, it's usually no, but they're interesting stories. And I feel like there's this very interesting intersection of detective stories and paranormal that is the X-Files, but could also be a novel one day. So let's just wait and see.Natalia Godyla:From your background, Emily, and your hobbies it seems you've got a lot of creativity either in writing or music. So what are your final thoughts on how creativity comes into play in the cybersecurity industry or in your day-to-day job?Emily Hacker:That's a really good question. And I think it's super important, especially in intelligence, which is all I can speak to because it's really all I've worked in in security. But one of the key aspects of working in threat intelligence is seeing a bunch of different data points. I might have a couple of data points here from open-source intelligence. I might see something weird on a machine and I might have an email and being able to connect the dots. And while that's not always something a machine can do, otherwise, we'd all been replaced by now. But it does require this level of creativity and this level of being able to remember, or kind of be like, "I wonder if I could connect this email to this thing that's happening with this machine."Emily Hacker:I was talking about detective novels earlier and I think that there's an aspect of that that kind of comes into play here too that's also an aspect of creativity, where you have to put the pieces together. You have to be able to see something once and then three days later when you have a malicious email in front of you be like, "Oh my God, this reminds me of this things from three days ago." There's also this level of creativity. I feel like that helps a lot of us. I was just talking about this with one of my coworkers yesterday, actually, about how one of the things that makes everyone on my team so successful, it is this level of, it's not by itself creativity, but I think it's an output for really creative people is this tenacity of when I see something I have to get to the bottom of it.Emily Hacker:And I think that I'm not just going to like run one query and be like, "Oh, computer told me it's X." I'm like, "But what is X? How do I get to the next part? What is it? How do I connect it to this Y over here? Do X and Y both connect over here to A maybe? Are they connected to this actor?" It's this level of just making a story out of the information that's presented to me that helps me, I feel like, be successful as an intelligence analyst. And I feel like there's a level of creativity to that that I honestly didn't think about until I've been in the industry for a while.Natalia Godyla:Yeah. I think you see a lot of unending curiosity with security folks as well. Like you said, as soon as you get one answer, it just opens up another question.Emily Hacker:Exactly.Nic Fillingham:So, Emily, you joined Microsoft in March of 2020, is that correct?Emily Hacker:Yes.Nic Fillingham:So you joined just as the mandatory work from home order was coming to place?Emily Hacker:Yeah. I've never ever been into the office.Nic Fillingham:Wow.Emily Hacker:Well, okay. I went into the office on day one to pick up my laptop and then went home, but I started after the work from home. So I've never met, well, I never met a lot of the people I work with in person. People always talk about the good old days of being on the office. Apparently there's a fridge that has bubbly water in it. One day I'll maybe drink some bubbly water.Nic Fillingham:It's a myth. It doesn't exist. We just tell that to people when they join the company and when they come in for the first time-Emily Hacker:Then they start and then they just make you work from home where you can buy your own bubbly water.Nic Fillingham:Yeah. Hey, where is this bubbly fridge? There's a fridge with bubbly water. No, it doesn't exist. You've been duped. So hang on. So I want to backtrack a bit because you talked about how you've got awesome colleagues and they've really helped you, so that's your experience completely through remote work.Emily Hacker:Yeah, it is.Nic Fillingham:So you've been able to join a new company, joined a new team, been supported and had sort of great experiences with colleagues through a hundred percent remote experience.Emily Hacker:Yep.Nic Fillingham:That's fascinating.Emily Hacker:I think one of the things that's been helpful is that there's a lot of new people on my team. So my team grew significantly around the time that I started. So me and another guy started on the same day and then four weeks later, another woman started and then over the summer we had two more people joined. And so we were in this together. And so it helped us. We all were in the same. It wasn't like everybody else knew each other and I was the new person, like, "Hey guys, let me join your conversation." We were all new. And so that helped a lot. But even the existing people on the team have been really, I don't know what word I'm trying to go for here, but they've been really open, I guess, to this remote work situation.Emily Hacker:The number of Teams calls, screen shares I've done where I'm just like, "Help. I don't understand what this means." And anybody I talk to is willing to sit on the other end of the Teams call and just walk me through what's happening. It has been honestly incredible. I'm really grateful for my team. I would like to go into the office one day, but I'd rather not be sick and I am glad that Microsoft is taking precautions. So considering the circumstances, things have definitely been going really well.Nic Fillingham:That's awesome. Well, Emily Hacker, thank you so much for being on Security Unlocked. We will work out how to send you a case of bubbly water.Emily Hacker:Thank you. Maybe then I won't go thirsty.Natalia Godyla:Well, we had a great time unlocking insights into security from research to artificial intelligence. Keep an eye out for our next episode.Nic Fillingham:And don't forget to tweet us at msftsecurity, or email us at with topics you'd like to hear on a future episode. Until then, stay safe.Natalia Godyla:Stay secure. See for privacy and opt-out information.
In this episode, hosts Nic Fillingham and Natalia Godyla speak with Mike Flowers and Cole Sodja of the Microsoft Protection Team, and Justin Carroll of the Microsoft Threat Intelligence Global Engagement and Response team, about how they’re using machine learning to identify and model lateral movement attacks. Then they speak to Dr. Anna Bertiger, Senior Applied Scientist at Microsoft, on how she’s using math to catch villains and make computer networks safer.  In This Episode, You Will Learn: What are lateral movement attacksHow machine learning helps address security challengesWhy grouping attack data can help better prevent threatsHow math is used to help analyze attack trendsHow AI and ML help identify patterns that can stop attacks  Some Questions We Ask: What are the most challenging parts of identifying lateral movement attacks?How does machine learning help understand how attacks would happen in the future?How do attackers change techniques as security techniques change?How do you use math to determine if an action is dangerous or benign?What is so beautiful about math? ResourcesMicrosoft Security BlogMike, Cole & Justin’s Blog PostTranscript(Full transcript can be found at Fillingham:Hello and welcome to Security Unlocked, a new podcast from Microsoft, where we unlock insights from the latest in news and research from across Microsoft security, engineering and operations teams. I'm Nic Fillingham.Natalia Godyla:And I'm Natalia Godyla. In each episode, we'll discuss the latest stories from Microsoft security, deep dive into the newest threat Intel, research and data science.Nic Fillingham:And, profile some of the fascinating people working on artificial intelligence in Microsoft security. If you enjoy the podcast, have a request for a topic you'd like covered or have some feedback on how we can make the podcast better.Natalia Godyla:Please contact us at or via Microsoft security on Twitter. We'd love to hear from you. Hi Nick. How is it going?Nic Fillingham:Hi Natalia, it's good. But bit of a first world problem here at Chateau Fillingham. I left a packet of dark chocolate covered mangoes open at my desk all throughout the weekend, and so now I'm inundated with the perfume of dark chocolate covered mangoes, which is a double edged sword. It's fantastic. And it's also terrible. It is better than the smell of acoustic foam, which you and I have both invested in, in order to make our microphone sound a little bit better. I will take dark chocolate covered mangoes over acoustic foam.Natalia Godyla:It sounds like I have something to learn from you. Maybe I should go get a bag of mangoes and open them on my desk, just leave them there. That's our podcasting Nic.Nic Fillingham:We're four episodes in and this is the wisdom that we have. We have devised being professional podcasters.Natalia Godyla:I feel like with that hot tip, we should dive into episode four.Nic Fillingham:This is episode four, which means we've had three episodes out there, which means people have been listening and downloading and rating and sending us tweets and emails with their feedback. And thank you so much to everyone that's listens, thank you so much everyone that has rated, that has sent us a tweet or sent us an email, we're reading every single one of them. We're actively following up on all of them and adding them to our editorial calendar for topics we can cover on future episodes. So thank you.Natalia Godyla:Yes, definitely want to second that. And with that, our first segment of episode four is a great one. We have three different perspectives on the episode. We speak with an expert on statistics and machine learning, and expert on threat analytics and expert on security research, as we dive into Microsoft 365 Defender and all of the great technology that underpins that product.Nic Fillingham:We learned a lot about Microsoft 365 Defender, and also how to interview three guests at once. That was some interesting logistical challenges, but I think we got there.Natalia Godyla:I think we earned our badge for it.Nic Fillingham:We unlocked an achievement with that one, and keeping with the theme of numbers and math, our expert that we're going to talk to is Dr. Anna Bertiger, who is a PhD and a post-doctorate in math. And I learned a new word, genuinely learned a new word, combinatorics, not even sure if I'm saying it right, but Dr. Anna will explain what that word is. Combinatorics, had you heard that word? I'd never heard that word.Natalia Godyla:I had not.Nic Fillingham:The teaser is that is a fancy word for counting things. And I'm quoting Dr. Anna there. We're going to learn how Dr. Anna approaches finding villains with math. That was a pretty cool conversation.Natalia Godyla:She's got an incredible passion for the intersection of those disciplines, and it's great to hear how she partners with security research in order to apply her knowledge of mathematics to our detections and to security as a whole.Nic Fillingham:Enough jibber-jabber, let's get on with the episode. Who says jibber-jabber?Nic Fillingham:So Mike, Cole and Justin, welcome to the Security Unlocked podcast. This is our first episode where we're going to be interviewing three guests at once. Thank you in advance for your time. I'd love to start, if you could just give a brief introduction of yourself, your role, what that means day to day at Microsoft. Mike, if we might start with you.Mike Flowers:Yeah, definitely. Mike Flowers and I am a security researcher within the Microsoft threat protection team, my day to day business is to try and bring together the different alerts that each of our sub components are exposing and bringing together into a single incidence that our customers can then look at to be able to get the whole picture of what's going on as part of an attack.Nic Fillingham:Great and Cole.Cole Sodja:Hi, my name is Cole Sodja. I'm a statistician. I work also in the Microsoft threat protection team. Role-wise, I primarily serve for helping implement machine learning for security applications, but pretty much what I spend my time on is one, collaborating with people like Mike and Justin, to understand kind of the threat landscape and identify attacks to model. Two, a lot of preparing and analyzing the data needed, so we could actually model those attacks appropriately using machine learning. And three, pretty much then just writing the code around the machine learning implementing itself, and then working with engineers to deploy it into our products such as MTP.Nic Fillingham:Excellent, welcome. And Justin.Justin Carroll:Hey, I'm Justin Carroll. I'm a threat analyst for the threat intelligence global engagement and response team. My role is essentially threat hunting typically across end point data looking for new or novel behaviors that are associated to known or suspected activity groups. Or new behavior that we may have interest in and providing intelligence on those behaviors that we're seeing, or new techniques to the different protection product teams to help inform them for detection opportunities or understanding what threats are doing and how the threat landscape is changing.Nic Fillingham:Excellent. Welcome to the podcast to the three of you. The three of you are co-authors on a blog post from June the 10th. That talks about how attack modeling is used to find and stop lateral movement in the MTP product, which has been recently renamed to Microsoft 365 Defender, MTP did stand for Microsoft threat protection. Mike, perhaps starting with you, I wonder if you could help kick us off with just an overview of what was discussed in this blog and what's an introduction to that technique.Mike Flowers:Definitely. So when we take a look at the different incidents across our customers, one of the things that we noticed is that when dealing with lateral movement, a lot of them had key characteristics that we could use to be able to bring together those different parts into a single incident. And so leveraging a lot of the real-world cases provided by Justin and also leveraging some ML models from Cole, we're able to bring all those signals into a single place. So that way our customers can take a look at those attacks in a single view.Nic Fillingham:Identifying lateral movement feels like it's obviously a very complex challenge, but also a pretty critical one upfront. What is obviously a basic question here, but let's cover it. What are some of the most challenging elements in identifying lateral movement as part of a bridge?Justin Carroll:I can at least speak to some of what we're seeing is a lot of the fact that the techniques used by the attackers, aren't really all that different from what administrators do. Most good attackers are trying to look like administrators when they're doing these attacks, or administrators that are great at their job, per se. Differentiating the legitimate behaviors that you're seeing that are associated with these protocols, such as SMB or WMI versus the malicious ones can be kind of challenging, because there is so much noise that you have to suss out quite a bit to infer, what is the main differentiator from this and sets it apart as far as malicious. And it's particularly challenging when you have multiple different machines and sometimes the attackers box isn't visible to you on telemetries, or you're only getting half the equation. So you're trying to piece together this multi part incident and figuring out all of it when you don't necessarily have the complete picture.Mike Flowers:One other key part about that is worth mentioning is that, a lot of times we'll end up seeing connections being made throughout a network whereby not only are they part of an attack, but they might not necessarily result in actual activity happening on the remote end. We'll see scanning, for example, happening in a network and in cases like that, the remote end won't actually have code execution on it, but it's still worthwhile to be able to see that type of telemetry. In that sense, what we're really trying to do is to bring together both components. So being able to see the type of telemetry, but also be able to bring in particularly, the instances in which a code execution does happen on the remote end.Natalia Godyla:How are we using ML to help solve some of these challenges?Cole Sodja:Really the challenge is how do you identify, or rather quantify legitimate behavior from the attack and that's where ML will help. There's two things we do. One is we do a form of supervised learning, where in essence, we create labeled data of attacks. For example, people like Justin will give us examples of actual attacks, that will provide some labels and then we'll take the data associated to these attacks and basically encode them in two features. Think of feature like the way we represent basically an attack, is in the form of a graph.Cole Sodja:The features form nodes of this graph, and features are stuff like, what are the network connections? What users are logging into the different machines? Are there any alerts on these machines? What are the different files that were dropped on the machine? What are the commands that are running on these machines? What's their parent child relationship? We take all these features basically, and we'll train a model to learn which combination of features actually correlate with the attack. In this case, we're looking at attacks that had an element of lateral movement. So we'll compute, what's the probability of observing lateral movement, given all these features and the examples we feed to the model. That's one way we use ML. The other way is through anomaly detection. Per what Justin was saying, where you have an administrator Cole Sodja:Who's making, let's say connections. We can build a model to learn what's normal for this particular account. Let's say that's an administrator making connections. How frequently do they make network connections to other machines? What do they do? Do they use scanning tools? How do they use the scanning tools? So we'll also employ anomaly detection, which is more unsupervised. We don't have labeled data there just to quantify what is normal. And that will also be used as an indicator to help basically filter out or remove the cases that are legitimate from what are actual attacks.Natalia Godyla:So, can you talk us through one of the attacks in the wild, the attacks that we're using to educate our ML algorithms?Justin Carroll:Yeah, so this attack is one that we've kind of seen quite often in the security space, more and more, is a human adversary hands on keyboard attacks, where they gain entrance to a network. Often for this group, for instance, it's typically through remote desktop brute forcing when they do this. So in this instance, what makes that a little bit difficult is they're typically brute forcing a local administrator account. So when they land on the machine, they are an admin, which gives them capabilities of tampering with antivirus solutions. It makes credential dumping very easily.Justin Carroll:They're not really restricted, right? In essence, if you are an administrator on a machine, you own that device and they don't have to, in this case, typically use many exploits or anything fancy. Once they have done the credential dump, as in the case that we saw, they can actually use those credentials typically with, if it's a server machine that they're landing on, at some point a domain administrator or somebody with some elevated privileges will have logged onto that machine, that's quite likely, and they can dump the credentials on that box and then use those credentials to continue their attack.Justin Carroll:Or other times what they'll actually do is look for password files and text documents, which is also quite common. And the attack that we found, they dumped credentials and then did significant scanning in the environment to find vulnerable targets with the main goal of distributing ransomware widely across the network. They then used a combination of sysinternals tool psexec and the windows management instrumentation, so WMI, to execute remote commands and code on the other devices in the network. And from that point on ransomed, many of the machines.Nic Fillingham:That was an example of an attack that you or your team found that had actually occurred. And then from that example, you were able to sort of perform a post-mortem and work out sort of what the attacker did. And then that formed intelligence that fed back into the machine learning model to sort of learn how these kinds of attacks would happen in the future. Is that correct?Cole Sodja:That's correct. So basically, we get examples like this. You can think of it as a cold start problem, where initially, we don't have any information or labels on this type of attack. Justin, for example, discover this attack, we get one or two labels, we're able to build a graph with these labels as an attack graph, essentially to start training the model, then what the model will do to continue learning, it will go use these nodes now we built in the graph, for example, as Justin said, prudential dumping to be a node in the graph, how they did lateral movement over WMI psexec could be another node of the graph scanning and so on, we'll build these graphs. Then the model, basically we'll go search historical data, looking for these nodes and bring back additional examples that the model feels is similar to this example.Cole Sodja:If it basically looks the same with high confidence, that is the probability that this exact attack is higher relative to any other attack out there or any other example of another attack, it will actually create labels itself and it will expand the graph based on cumulating, additional information in the graph. In those cases, if we didn't have those as nodes, the model will actually add those as nodes and it will keep that and then compute probabilities of those. And again, if there's higher likelihood that those nodes are associated to real attacks or attacks like this, it will retain them. And if not, it will then learn how to filter them or compute them as very low likelihood. And they won't receive a lot of weight in the actual construction or prediction. So, that's how we train the model through these examples.Natalia Godyla:And what's new about this technique? What were we doing before this technique, or what were security operations teams doing before?Cole Sodja:So, before this technique was available, each of the alerts that was happening on the different devices were silent. So along those lines, if a ransomware attack happened within an organization that had, let's say, 10 devices, that each of those 10 devices would have separate alerts in them based off of what they're able to detect. And what we're trying to do is to bring together all 10 of those incidents into a single one. So, that way you can go into that one place to take a look at it all together.Natalia Godyla:So one centralized location, that makes sense. And what's next for the team then as you evolve the product?Cole Sodja:Oh sure, from an ML side, there's two things. So one, we already started to work on and have some success, but it's ongoing. It's currently not implemented in our product. It's more of a proof of concept or pilot right now is classifying threat actors, like the [inaudible 00:16:53] example. So when we see attacks, rather than just correlating what we observe in the attack, we could actually start computing the probability that this attack is this known threat actor. And given that, we could start asking questions like what's the probability given that we believe it's this threat actor that we're actually going to see ransomware in the coming stage of the attack or some other type of objective from the attacker. So those are two active areas of research and stuff we plan on integrating into our product at some point in the future, it's the classification of threat actors that we're tracking and predicting the attack stages.Cole Sodja:What's going to come next in the attack, given the intel we have about the threat actor, that's one. The other one is basically expanding these types of correlations beyond lateral movement. We've had quite a bit of focus recently on human operated ransomware, but there's other things we plan on doing, integrating or extending this type of framework for better correlations of these alerts that still are hard to correlate and end up in silos. So it's something we want to extend this framework to just better correlate alerts that are probable as part of the attack, but we can't infer it like deterministically.Nic Fillingham:I'm just wondering, are you seeing adversaries change their approach or their techniques in response to the success really, I guess, of these new tools and techniques in the product?Justin Carroll:Yeah, you'll frequently see them changing techniques depending. So it kind of depends. Adversaries most often use what works and typically in lots of instances where they're trying to deploy, for instance, ransomware on numerous targets quickly, right? Like they want to have high confidence that something's going to work. So in that instance, they only change techniques when they hit a roadblock, essentially when that no longer becomes valid or they're being stopped too quickly in their attack to fully execute it. We have seen quite a few different adversaries actually specifically switch to different techniques and have registry files named, things that basically indicate frustration with the way our products are stopping them. So they get very frustrated at Defender for instance, so they will try and use different tools and actually maim them, as you can tell that they are quite antagonized by how we are constantly monitoring them and trying to stay basically one step ahead of them.Nic Fillingham:Damn you, Defender WXA, something like that?Justin Carroll:A little bit more explicit, but yeah, a lot more explicit. But yeah, so we do see them modify quite a bit, but it kind of depends. I know with some of the more recent threat actor tracking that Cole and I and Mike have been kind of digging in and working on, we kind of see a slow progression over time where you'll see some techniques kind of drop off for a bit and then eventually, sometimes they resurface again, it just kind of depends on what is the most effective for them to get their job done.Nic Fillingham:And it feels like utilizing machine learning as a tool here in this process has one of the additional benefits there to your point, Justin, is if an attacker decides to revert back to... Obviously it's good at identifying variations on attacks, but if they want to revert back to something they haven't done in many, many years, you're not asking a human analyst to then dig back into their dusty cobweb memory bank. The machine learning model has that they're sort of somewhat instantaneouslyJustin Carroll:The advantage of ML as well, tied to what you're saying, as far as understanding that old techniques are still part of the model. So it knows how to handle them. Most attackers typically aren't altering all of their techniques, right? It's different sub components of the attacks so that either have been made more difficult by different product changes or things like that. The advantage of the ML is you're able to find those attacks where overall 70 to 80% of it is the same. And then you can use that surfaced information to know what they've changed to then put it back into the model to continue to modify with that. So unless the attacker completely changes from the ground up, which often they just don't do, you have a really good method of kind of keeping your finger on the pulse.Natalia Godyla:So it's actually benefiting us in a way, because we're able to just continue to evolve the ML model because we already have that base data and can just adjust based on the subtle changes. Is that accurate?Cole Sodja:Yeah, that's accurate. I think Justin stated it nicely. Those are really the benefits because if an attacker completely changes everything of course, which would mean we don't have any previous features to even leverage on, to start computing a probability, that's a different story. But since that's extremely rare, that's where ML is quite useful. It can continue as I said, to grow and shrink this graph and dynamically learn these probabilities and through surfacing the probabilities, we could rank them accordingly. Cole Sodja:And that's where people like Justin could go look at it and say, "Oh, okay, yeah. We think with, let's say 65% confidence right now, it's still the same actor, but here's some new things that the ML model discovered as part of this attack" that then Justin could look at and then basically further interpret and help give that feedback back to the algorithm. So then it understands what these new features are, and how they are related. Giving that context essentially to the model, I would say is key.Nic Fillingham:And is there anything that the customer needs to do, or the individual Security analysts or practitioners need to do, to take advantage of this technique? Or is it just sort-of baked into the product?Mike Flowers:And these things are baked into the product. So, whenever a customer pulls up their list of incidents, they had to look at if there are any that span multiple machines, And if they contain alerts that have cited a lateral movement activity in them, then they'll automatically be brought into that single incident for them to be able to take a look at.Natalia Godyla:How did the model originate? What was the driver for this coming to light?Justin Carroll:A lot of how it came about was just a need on the analyst part of having a model to basically combine a wide set of disparate signals that at first glance may not appear related and required a significant amount of work to correlate all the behaviors into a meaningful fashion to understand that they were tied to one specific incident, or one actor. It came about organically as data science is one of the perfect partners for security to empower each other and then working together to continually build new models and then using those models to help inform the analysts of new behaviors and allowing them to quickly find interesting incidents that may drive the intelligence conversation or understanding where we have a product alerting opportunities. It's a very natural collaboration that is extremely effective.Cole Sodja:I will just add one thing to that. So, one thing data science brings, it's not just like the methodologies, if you will, in terms of how we design the right tool for the jobs, there's an exploration phase. So, one thing like Justin was mentioning is you have this huge space of signals to search through, and yeah, we have some previous examples and there is also what we like to call the "unknown unknown," stuff we haven't seen, even the threat experts missed. For example, because they are kind of weak signals in themselves. So, it's searching through this large dimensionality and then correlating them all and returning essentially what a model or what the scientist believes is to be indications of attacks that we might have missed, or a part of an attack that we capture, but we didn't completely get the whole story of the attack.Cole Sodja:And that's where that collaboration becomes quite natural. So, we will explore, then we'll go back and have a discussion. We'll review, and that will be feedback into how we further explore and we'll keep going, generating new examples from that, and so on. Eventually, that will lead to the definition of the model, actually.Nic Fillingham:There's almost always massive, massive numbers behind the scenes here, and I know a lot of our audience like to learn or to hear about the immense scale that's happening behind the scene. Anyone got a big number you want to throw at us to impress as to the scale and output of what this can do?Mike Flowers:We do generate tens of thousands of alerts every single day for our different customers. And what I find to be particularly awesome about the work that we've done with this project is bringing together, or picking out those alerts within that giant set, to be able to filter it down to the select 30, 40, 50 alerts that are part of a single incident that's happening within a given work and making it so, that way we're able to classify it.Mike Flowers:So, that way it's all part of one attack and bring it together for the end analyst. So, I would say taking that number hundreds of thousands, even of different alerts across the entire timeframe and taking out the less than a hundred that are relevant to this specific attack.Natalia Godyla:Great. Well, thank you for that. And thank you, Cole, Mike, Justin for joining us today, it was great to walk through all the great work you're doing.Cole Sodja:Thank you. My pleasure.Justin Carroll:Yeah, thank you.Mike Flowers:Thanks. Happy to be here.Natalia Godyla:And now let's meet an expert in the Microsoft security team. To learn are more about the diverse backgrounds and experiences of the humans, creating AI and tech at Microsoft.Nic Fillingham:Doctor Anna Bertiger, thank you so much for joining us. Welcome to the Security Unlocked podcast.Dr. Anna Bertiger:Thank you so much for having me.Nic Fillingham:If we could start with, what is your title and what does that really mean in sort-of day-to-day terms? What do you do at Microsoft?Dr. Anna Bertiger:So my title is Senior Applied Scientist, but what I do is I find villains.Nic Fillingham:You find villains, so how do you find villains?Dr. Anna Bertiger:So I find villains in computer network. It's all the benefits of a job as a superhero with none of the risks, and I do that using a combination of security expertise and mathematics and statistics.Nic Fillingham:So, you find villains with math?Dr. Anna Bertiger:Yes, exactly.Nic Fillingham:Got it. And so, let's talk about math. What is your path to Microsoft, because I know it heavily involves math? How did you get here? And maybe what other sort of interesting entries might be on your LinkedIn profile?Dr. Anna Bertiger:So I got here by math, I guess. So, I come from academic mathematics. I have a PhD in math, and then I had a postdoctoral fellowship in the Department of Combinatorics and Optimization at the University of Waterloo in Waterloo, Ontario, Canada.Nic Fillingham:Could you explain what that is, because I heard syllables that I understood, but not words?Dr. Anna Bertiger:So that, is the Department unique to the University of Waterloo. So, Optimization is maximizing, minimizing type problems.Nic Fillingham:Got it.Dr. Anna Bertiger:And Combinatorics is a fancy word for counting things.Nic Fillingham:Combinatorics?Dr. Anna Bertiger:Yeah. Which you can do in fancy and complicated ways. And so, so that's what I did when I was not going to make mathematician is I counted things in fancy and complicated ways that told me interesting things frequently about geometry. And, then I decided that I wanted to see the impact of what I did in mathematics, in the real world, in a timeframe that I could see. And, not on the sort of like you think of beautiful thoughts, it's really lovely.Dr. Anna Bertiger:It's a lot of fun. And, then hopefully someone uses them eventually. And so, I looked for jobs outside of academia and then one day a friend that Microsoft sent me a note that said, "If you like your job, that's great. But if you don't, my team wants to hire somebody with a PhD in Combinatorics." And I said, "That's me!" And so, it took a while I flew out for an interview, they asked me lots of questions. I, when I'm interviewing for a job, I evaluate how cool the job is by how cool the questions they ask me are. They ask me interesting questions, that's a good sign. If they ask me boring questions, maybe I don't want to work there.Nic Fillingham:Do you remember any of the interesting questions? Anything stick out?Dr. Anna Bertiger:Yeah. So they asked me that team was involved in the anti-credit card fraud system at Microsoft. So, someone is typing your credit card number into Microsoft's website. Are you going to call up and say, that was fraud? If the answer is yes, we don't want to, we don't want to complete that sale. If the answer is no, then we would like your money. And so, they asked me a bunch of questions about how you get the right data for, for credit card fraud. So like how, how do you know, how do you get a bunch of labeled data for credit card fraud that says this is fraud, this isn't fraud.Natalia Godyla:Was there something that drew you to the cybersecurity industry? When your friends showed you this job, did you see security and go, "Yeah, that's cool."Dr. Anna Bertiger:So, I didn't actually see security in that job. Like that team didn't only work on fraud. We worked on, we also worked on a bunch of marketing related problems, but I really loved the fraud, related problems. I really loved the adversarial problems. I, I like having an adversary. I view it as this like comforting, friendly thing that like you solve the problem. Don't worry. They'll make you a new one.Nic Fillingham:Haha.Dr. Anna Bertiger:It's true.Nic Fillingham:So, hang on so, you, you go to bed at night, and sleep soundly knowing that there are more villains out there?Dr. Anna Bertiger:I mean, I would kind of like to get rid of all the villains, but also like they're building me some really cool problems.Nic Fillingham:Yeah, you're a problem solver in there throwing some good challenges at you.Dr. Anna Bertiger:Right. I'll like make the world a better place. School of thought. I would like them all to disappear off the face of the planet on the like entertaining me portion. Their problems are pretty good. And so I worked a bunch on, on credit card fraud related problems on that team. And at some point a PM joined that team who had a, who was a cybersecurity person who had migrated to fraud. And I said, "Well, I'm not a cybersecurity person." He said, "Oh no, you are. It's a personality type. And it's you." And then I worked at some other things, worked on some other teams at Microsoft, did some windows quality-related things. And I, it just wasn't as much fun. And I found my way back to cybersecurity. And I've been here since.Natalia Godyla:And how do you apply your academic background to that role today? What do you see transfer the most?Dr. Anna Bertiger:So, I think a lot about mathematics and statistics on graphs. So, maybe it's networks of computers and I'm looking for surprising connections, that's something I think about a bunch and surprising connections might be that people are weird, or it might be that someone who doesn't know your network and doesn't behave like the people who are usually in your network are, is they're making connections between computers and that is lateral movement. So that, suggests there's some advanced human actor in your network.Nic Fillingham:So how do you use math to determine if it's just, "Oh no, this person is doing something funky, but benign." Versus bad actor, a lateral move.Dr. Anna Bertiger:So that, is sort of the secret sauce of cybersecurity expertise. So, I, the math tells you, this is weird. This is not typical, but the math doesn't tell you whether it's good or bad, the math just tells you it's atypical. And, so then, you hope to look for atypical along an axis where atypical is likely to also be poor behavior, is likely to also be someone malicious. And that, is about working with people who are cybersecurity experts, working with threat researchers, working with threat intel, and trying to find the right access to work along for. Oh yeah. If it were weird in this way, that's probably bad. And you talk to them, you try something, rinse and repeat, a lot.Natalia Godyla:How do you use AI or ML's tools to solve some of these problems?Dr. Anna Bertiger:So the AI and ML is about learning what's normal and then when you say, Hey, this isn't normal. That might be malicious. Someone should look at it. So our AI and ML is human in the loop driven. We don't act on the basis of the AI and ML the way that some other folks might and there's certainly security teams that have AI and ML that makes decisions and then acts on them on its own. That is not the case. My team builds AI and ML that powers humans who work in security operation centers to look at the results. And so I use ML to learn what's normal. Then what's not normal, I say, Hey, you might want to look at this because it's a little squiffy looking and then a person acts on it.Nic Fillingham:So what are some of those techniques, AI and ML, obviously very broad terms, they could have quite a wide scope, what are some of the techniques or approaches that you use mostly? Is that even an answerable question or do you use everything in the tool belt?Dr. Anna Bertiger:I mean, I most prefer the technique that solves the problem, but that said, I do have favorites. And so I use a lot of statistical modeling to figure out what's normal. So fit a statistical distribution to some numerical data about the way the world is working and then calculate a P value, that you might remember from stat one, if that's something you've done, to say, oh yeah, well, there's only a 10th of a percent chance that this many bytes transferred between these pair of machines under normal behavior. Someone should look at that. That's a lot of data moving. And there, I like to use a group of methods called spectral methods. So they're about if I have this graph, I have a bunch of vertices and I can have edges between them. I can make a matrix that has a one in cell IJ. If there's an edge between vertex I and vertex J. Let me know if I'm getting too technically deep here.Nic Fillingham:You are, but keep going.Dr. Anna Bertiger:And then now I have a giant matrix. And so I can apply all the tools of linear algebra class to it. And one of the things I can do is look at its eigenvalues and eigenvectors. And one way you might sort of compress this data is to project along the eigenvectors corresponding to large and absolute value. Eigenvalues. And now we can say things like, all the points that are likely to be connected end up close together and we can try and learn something about the structure, the network, and what's strange. And we've done a bunch of research in that direction. That is stuff I'm particularly proud of.Natalia Godyla:So I know you mentioned this is very human in the loop, so you're bringing this to somebody and they now have the information that they can make a determination based on. What about plugging it back into the Microsoft Solutions? Are we using this information to inform our products as well? Or are you focused really on empowering our security folks?Dr. Anna Bertiger:Well so, here, the security folks are our customers. This is the product we are selling them is the alerting us that something is wrong, products. So sometimes it's security folks at Microsoft, I've written things that went to the hunters that power Microsoft Threat Experts that they look at and say, eh, not so much, Anna, or sometimes, this is really gold. I mean, and they have more tolerance than many for, well, it can be lousiest sometimes as long as it's gold, sometimes also. And then, also I've written things that go to our customers via the products we sell.Natalia Godyla:What are you most interested in solving next? What are you really passionate about?Dr. Anna Bertiger:I'm really passionate about two things. One of which is sort of broadly speaking, finding villains, finding bad guys. So part of what I do is dictated by what they do, right? They change their games, I have to change mine too. And then also I have a collection of tools that I think are really mathematically beautiful, that I'm really passionate about. And those are spectral methods on graphs and sort of graphs in general. And so I'm really passionate about finding good applications for those. I'm passionate about understanding the structure of how computers, people, what have you, connect with each other and interact and how that tells us things about what is typical and what is atypical and potentially ill-behaved on computer networks and using that information to find horrible people.Dr. Anna Bertiger:I think I've stopped being surprised by what our adversaries can do, because they are smart people who work hard. Sometimes I'm disappointed in the sense of, damn, I thought I solved that problem and they're back. But, I mean, and that's mostly just, you feel like the sad balloon three days after the party.Natalia Godyla:At the end of the day, why do you do what you do?Dr. Anna Bertiger:I think there are two reasons I do what I do. The first, which is, I want to make the world a better with the ways I spend my time. And I think that catching horrible people on computer networks makes the world a better place. And the other, which is that it's really just a ton of fun. I really do have a lot of fun. We think about really cool things, neat concepts in computing, and beautiful mathematics. And I get to do that all day, every day with other smart people. Who wouldn't want to sign up for that?Natalia Godyla:You've called mathematics beautiful a couple of times. Can you elaborate? What do you find beautiful about math? What draws you to math?Dr. Anna Bertiger:I find the ideas in math really beautiful. And I think that's a very common thing for people who have a bunch of exposure to advanced mathematics, but isn't a thing we filter to folks in school as well as I would like. That if you think about the Pythagorean theorem. So that's a theorem that most people learned in high school geometry that says-Nic Fillingham:I know that one.Dr. Anna Bertiger:The square of the lengths of the two sides of a right two legs of a right triangle equals the sum together equals square, the hypotenuse length. And if you-Nic Fillingham:Correct.Dr. Anna Bertiger:That is a fact, okay? And if you learn it as a piece of trivia, then you go, okay, that's the thing that I need to know for the test and you write it down and you put it on a flashcard or whatever. But what I think is really beautiful is the idea of how do you think that up? And the sort of human ingenuity and figuring out that that's true and the beautiful ways you can show that that is true, for sure. There are some really, really beautiful ways to be able to prove to yourself that that is true.Nic Fillingham:And is that math or is that human ingenuity? Is that the human mind, is that sort of creativity or is it altogether?Dr. Anna Bertiger:It's sort of both. I mean, the things that I love about math are the creativity and the new ideas. And so to me, those are very wrapped together and, sort of, math is as much about there's some saying about truth and beauty, and math is about those things.Nic Fillingham:Changing topics, sort of slightly, are you all math all the time? Do you have a TV show you're binging on Netflix? Do you have computer games you like to play? Are you a rock climber? What's the other side of the math brain?Dr. Anna Bertiger:So the other side of the math brain for me is things that force my brain to focus on something that is entirely not work. And so I really love horses and I have a horse and I love spending time with her. And I love riding her. She's both a wonderful pet and just a thrill to ride.Nic Fillingham:What's her name?Dr. Anna Bertiger:I call her Elsa, but on paper, her name is Calloway's Blushing Bride.Nic Fillingham:Wow.Dr. Anna Bertiger:I didn't give her either of those names.Nic Fillingham:Do you think of horse riding in mathematical terms? Do you sort of think about velocity and angles and friction and all that kind of stuff?Dr. Anna Bertiger:No.Nic Fillingham:Or is it-Dr. Anna Bertiger:No.Nic Fillingham:Just organic?Dr. Anna Bertiger:I really think about horseback riding in terms of sort of what it feels like. It's the opposite of sort of dry and technical.Nic Fillingham:Awesome.Natalia Godyla:Well, Anna, it was a pleasure to have you on the show today. Thank you for sharing your love of math and horses and hopefully we'll be able to bring you back to the show another time.Dr. Anna Bertiger:Thank you so much for having me.Natalia Godyla:Well, we had a great time unlocking insights into security from research to artificial intelligence. Keep an eye out for our next episode.Nic Fillingham:And don't forget to tweet us @msftsecurity or email us at with topics you'd like to hear on a future episode. Until then, stay safe.Natalia Godyla:Stay secure. See for privacy and opt-out information.
In this episode, hosts Nic Fillingham and Natalia Godyla speak with Hardik Suri of the Microsoft Defender ATP Research Team about using behavior-based detection and machine learning to block attacks against Exchange servers, and why it’s so critical to patch and enable security capabilities.  Then they speak to Dr. Karen Lavi, a Senior Data Science Lead in the Microsoft Defender Research Team, on the neuroscience of threat detection, and how her team is using AI and machine learning to predict and prevent malware attacks.  In This Episode, You Will Learn:  Why Exchange servers are so vulnerable The best way to defend against web shells The simple things security professionals can do to protect under-protected servers How neuroscience factors into threat detection How to catch ‘patient zero’ in an attack Some Questions We Ask:  How are techniques for detecting and blocking attacks evolving?  What’s next for behavior-based blocking?  How does machine learning benefit security?  How do you build a diverse team to catch threats?  What’s the next innovation in security research?  ResourcesMicrosoft Security Blog   Hardik’s blog post transcript can be found at Fillingham:Hello and welcome to Security Unlocked, a new podcast from Microsoft, where we unlock insights from the latest in news and research from across Microsoft's security engineering and operations teams. I'm Nic Fillingham.Natalia Godyla:And I'm Natalia Godyla. In each episode, we'll discuss the latest stories from Microsoft security. Deep dive into the newest by Intel, research, and data science.Nic Fillingham:And we'll follow some of the fascinating people working on artificial intelligence in Microsoft security. If you enjoy the podcast, have a request for a topic you'd like covered, or have some feedback on how we can make the podcast better.Natalia Godyla:Please contact us at, or via Microsoft security on Twitter. We'd love to hear from you.Nic Fillingham:Hello listeners, welcome to the third episode of Security Unlocked. And hello to you, Natalia. It is October. The leaves are turning. It is the time of year of candy corn and high fructose corn syrup. I know you love October.Natalia Godyla:I do. I am all in for pumpkin spice lattes and cyber security awareness month.Nic Fillingham:A match made in heaven. If they reformed the Spice Girls, you should audition to be the sixth girl called Pumpkin Spice.Natalia Godyla:Two out of 10 on that joke.Nic Fillingham:Two out of 10? I thought it was pretty good. Anyway. Cyber security awareness month, is that a Microsoft thing? Is that an industry thing? What's all that about?Natalia Godyla:It's an industry thing, but Microsoft is definitely invested in doing their part during this month. So it's really exciting to see everyone empowering the cyber security world to get the word out, which is what we do on this podcast too.Nic Fillingham:Exactly right. I was just going to say, Security Unlocked, the podcast. Every episode we're about helping spread the word of the importance of cyber security and helping empowering our listeners with more information about how all this stuff works.Natalia Godyla:Yeah. I'm excited for this episode. We've got a great lineup.Nic Fillingham:First up, we talk to Hardik Suri on the importance of keeping servers up-to-date. But more importantly, or more specifically, how he and his team are working on behavior based monitoring to protect, what's referred to sometimes as, under-secured or under-protect servers.Natalia Godyla:Yeah. And we talk to Dr. Karen Lavi about her background and how she came to cyber security through a really interesting journey. She's been a medic. She's been in the Israeli Defense Forces. All with the intent of just doing good, and she'll talk to us about her perceptions of AI and how neuroscience and the rest of her background connect to cyber security.Nic Fillingham:It is a great conversation. It's a great episode. I hope you'll enjoy it. Let's get on with the pod.Nic Fillingham:Hardik Suri, welcome to the Security Unlocked Podcast. Thanks for joining us.Hardik Suri:Thank you for having me.Nic Fillingham:Could you start by just introducing yourself, telling us about your role at Microsoft, and what you do day-to-day?Hardik Suri:Sure. I work as a senior security research with the Microsoft Defender ATP research team. I'm currently based in Vancouver and my daily responsibilities is getting up to speed on all the latest threats which are out there, and just checking which ones are impacting Microsoft products. And if they are, how to be durably detect and protect these latest advanced attacks. Anything which touches the endpoints would be under our radar. It could be explorers, it could be malware, it could be an email with an attachment. When the user clicks, it downloads itself and does all those funny things. So any suspicious or malicious activities happening on the endpoint we get visibility as the product, and then we try to see how we can detect that part and prevent that from abusing a lot more.Natalia Godyla:So your research process starts with the signals coming from the product? And then, when you find something that is suspicious or interesting that is your jumping off point to dig in further?Hardik Suri:Mm-hmm (affirmative). There are two ways. Either we proactively go and find things and then we come back and see if we've got something in telemetry. Or if telemetry can give us something interesting, and then from there we can pivot and find what really happened.Nic Fillingham:Hardik, you authored a blog post in June called Defending Exchange Servers Under Attack, which we'd love to talk about. Could you walk us through what you discovered and how you addressed it in this June 24 blog?Hardik Suri:How it all started was we had telemetry on a piece of code called web shells on these exchange servers. Web shells are nothing but they are a comparatively small piece of code which attackers can install on these servers and then can control the exchange server, in terms of running commands, or dropping more binaries, or moving laterally. So that piece of code on the servers is critical to the attacker. And if you find any instance of that piece of code, we know that the server is already compromised and the attacker is already operating on that server. That was the starting point, and then when we look at the server in detail, we could see what all actions the attacker did. Hardik Suri:Whether it was doing reconnaissance activity that is trying to enumerate all the entire organization and the users and finding which critical accounts he should target, or is it like dumping credentials, like if we can get credentials, we can move laterally on the organization and impact or infect more machines. So that was the starting point where we see a web shell installation. That's alarming a lot for us, and then we would deep dive and see what all the attacker did.Natalia Godyla:Were there challenges in detecting this threat?Hardik Suri:Yes. It's not your typical endpoint infection. When we say these servers getting compromised, these servers already have a lot of embed tools which attackers can abuse. So they don't really have to bring their own tools, which can be easily detected. But if they're already using existing tools and scripts, which are used by admins, then the traditional problem comes where how do you detect if this is an activity done by an admin of the server or is it an attacker doing the activity? Hardik Suri:So the whole challenge was to separate out the clean, or the noise, and just focus on the malicious part, which the attacker did.Nic Fillingham:I wanted to ask about web shells. You talked about it being relatively small pieces of code. Let's just explain that a little bit. A web shell is a piece of code that exposes the shell of the system to the web, is that accurate?Hardik Suri:Yes. How this works is a web shell is nothing but a small piece of code which exposes functionality to execute code on the endpoint. So how you would install a web shell would be these exchange servers have folders which are accessible over the internet. So if you can install that piece of drop, that piece of script there, and visit that URL, you can control that script, and you can pass the command you want to execute as HTTP, URL or it could part of the code keys or any HTTP section. And internally, the listening script would get that piece and then execute it on the behalf of the server. So how a process tree looks is the exchange server, the instance of the exchange server, is actually executing these commands on behalf of the attacker.Natalia Godyla:How are we evolving our techniques for detecting and blocking, especially when it comes to evasive technologies, like web shells to evade the file based protections?Hardik Suri:File based protections would not be durable, a longterm solution for this. So what we did was we started profiling the behavior activity of these exchange servers. And understanding clean activities, or clean behaviors you would see in an exchange server, which helped us in eliminating the noise we were seeing in these attacks. So Microsoft Defender has this powerful behavior competence where it can inspect the behaviors initiating from these exchanges processes. And then we can see what kind of activities these are doing. Hardik Suri:And based on that, we can, with some confidence, say if it's trying to spawn. Let me give you an example. If an exchange server is trying to spawn CMD.EXE, or MSHDA, or these known suspicious system files, then it's highly likely that it's been compromised and there's a web shell on the server.Nic Fillingham:And are these behavior detections, are they rule based, or are they a bit more dynamic? Are they taking into account other factors and maybe more machine learning based determinations?Hardik Suri:Mm-hmm (affirmative). Yeah. They are very generic in nature. And we do take input from machine learnings. All these behavior patterns, they're getting fed into the cloud behavior machine learning models. And what that helps us is the machine learning model can then provide blockings advisors to these endpoints, where even if the endpoint behavior competence is missing something, the machine learning can cache that based on its intelligence, and still block the attacks.Hardik Suri:We have this pattern technology, where we have these behavior patterns which we just collected from the endpoint, and it's getting fed to the machine learning. And the machine learning is getting more intelligent and can actually block future attacks.Natalia Godyla:What do you recommend security practitioners do in order to protect against the exchange server attacks?Hardik Suri:Sure, yeah. The first thing, the most common one is to apply patches. Since these machines are very critical to the business, so it's a common saying that if exchange server goes down, the business goes down. So applying later security patches is the top priority the admin should take. The reason for this is that we are finding Hardik Suri:... A lot of exploits vulnerabilities in exchange servers which can be exploited and which can allow the attackers to land on these exchange servers directly, which is a game over for the organization. So, being proactive in applying patches is certainly the topmost. Second, keeping the security solutions up to date. So, don't turn off your antiviruses, your firewalls, your network protections, keep them on and keep them updated.Hardik Suri:There has been a myth where the admins would disable the security products so that they don't really interfere with the critical workings. But what is required here is a more intelligent understanding of what settings to turn on and what settings to not turn on if they are actually interfering. So, just out-rightly turning off the security solutions is not recommended and would open the door for more exploitations. And restrict access and follow something called principle of least privilege with credential hygiene. So, keep all privileges to the lowest and the ones which are really required, keep them at certain privileges. Avoid using highly critical credentials across machines.Hardik Suri:And finally, prioritize alerts, I'll say. Since all the organizations have some sort of central logging capabilities where they can see all the alerts coming in, any alert from the server should be considered a high priority and should be investigated thoroughly. This would help in limiting the impact because from my experience, the time the attackers come into the system, they would spend days on just doing reconnaissance on the systems. They would not jump on executing things. They would just be there and enumerate the users and try to understand the environment. That would take days and if we can identify and detect and block them at that stage, that would really limit the harm they can cause.Nic Fillingham:So, moving forward is some of the work that you've done here and you talk about in this blog, is that actually going to help customers that have exchange servers where they may not have both of these things where they haven't applied the latest security updates or they have turned off or greatly minimized some of the security features, is some of this behavior monitoring, that you talk about in this blog, is that an additional layer of protection?Hardik Suri:Oh yes. That's an additional layer I'll say. And a more durable layer. The attacks on exchange servers are very different from attacks on Endpoint where you would not see the attackers bringing their own malicious binaries, which would get detected by this antivirus office. They will generally rely on the tools which are already there on these exchange servers.Hardik Suri:For example, if they land on exchange servers, one of the things they would want to do is dump the emails, because the servers are known for containing all the organization emails. For dumping and ex-filtrating these emails, they don't really have to bring any tool of their own. There are commands already installed on these servers where they can just run them and get all the emails.Hardik Suri:So, while the file-based or the traditional antivirus solutions may not detect these attempts. The behavior components can surely detect this where you would see email getting dumped and then getting zipped and then getting ex-filtrated. All these different events we can correlate together and then piece a picture together where this could potentially be an ex-filtration of corporate emails. So, that adds a lot of value and a lot of protection.Natalia Godyla:And what's next for the behavior-based blocking? I recall in the blog, you had outlined that there are ways in which the threat actors are starting to evade our detections. So, one example that you gave was Mimi Cats. Mimi Cats could be blocked, but there's a different way that they could leverage Mimi Cats or wrap the program in order to get past our detections again. So, I'm sure it's like a cat and mouse game where you're continuing to evolve the product while they're continuing to evolve their techniques. So, what's next?Hardik Suri:Doing more investment on these servers is something in the pipeline. Like you rightly mentioned, attackers would always play a cat and mouse game with files where we would detect something and they would modify that and then we stop detecting that. That's where the behavior component is so important. The cost of changing a behavior is much more. Behavior translates to a technique. So, the effort to create or use a new technique by an attacker, the cost of that is much more than simply wrapping a binary or adding some or removing some bites, where it's the protection. So, the whole point is to how to increase the cost on an attacker to execute an attack. And while we sit in a more generic layer where they might evade file-based detections, but for them to really evade us completely, they have to create a new attack from scratch, which we have seen that the attackers won't do. They would generally want to reuse whatever they have created on different organizations.Hardik Suri:So, the behavior component will always be a much more durable way of protecting customers, I'll say.Nic Fillingham:Hardik, was there an ah-ha moment for you and or your colleagues when you were going through this process? Did a particular piece of data or telemetry allow you to see the big picture in a dramatic way or was it a slow drip?Hardik Suri:Well, it was a slow drip I'll say, because the attackers like I said, it's not a typical Endpoint detection where the whole detection is over in a few seconds or minutes. These attackers are in your organization for weeks and months before they start doing anything malicious. So, we need to be patient and we need to be watching them all the time and lay traps for them, if they do something, we get telemetry and we block them outright.Hardik Suri:Well, the ah-ha moment was when they were trying to abuse this thing called Exchange Management Shell. That's a very critical piece of platform, which the admins use to maintain the exchange servers, and a few of the actions could be exporting mailbox emails or migrating them. While we could see the attackers doing your typical activity of reconnaissance and that credential dumping. The moment we saw the attackers going after the Exchange Management Shell, and trying to dump the emails, that was the point we could really understand the motive of the attackers and we could also see what kind of emails they were looking for. They were searching for specific subjects. They were searching for certain strings in the body. So, we could really understand the mindset of the attackers and what they were actually after.Natalia Godyla:So, what was the end result of the attacks then? So, they ex-filtrated credentials as a result, what did they do with them?Hardik Suri:So, they were really after the corporate data or the content in those emails and they were trying all their effort to how they can dump and ex-filtrate this part because the exchange servers would contain all the critical information and emails would be one of them. We did see them moving to other machines where they could find more information, but if we keep focused on the exchange servers, the emails were what they were after.Natalia Godyla:Was this with the goal of selling this data or compiling a large dataset to use for other malicious intent?Hardik Suri:These are really advanced attackers. So, these attackers would generally use this kind of information to gain more information on the organization. It could be your typical corporate espionage cases or IP theft cases where they would want to collect all the IP, Intellectual Properties of an organization. We are not sure at this point how they use that data, but that seemed like the intention based on the strings they were searching inside the emails.Nic Fillingham:The behavior modeling that's happening on the Exchange Server and then the machine learning that's up in the Cloud, can any of those learnings, those behaviors, that learning, can that flow into other models to help protect other servers that are open to the web? I'm wondering if some of the work you've done here is going to filter out and benefit other products and services?Hardik Suri:Oh yeah. Certainly. So, the ML model is quite generic and it doesn't really serve exchange servers only. It provides protection for all the endpoints. So, if we detect something on one endpoint and that same technique is used on exchange servers, if the Cloud already has that information, it could have out rightly blocked that.Hardik Suri:So, it kind of collects everything and doesn't really differentiate between what endpoint that is. And malicious is malicious, it doesn't really matter if it's on an exchange server or an Endpoint.Nic Fillingham:Hardik, what do you do when you're not a security researcher? What do you do for fun?Hardik Suri:So, I'm a musician. I play the guitar. Back home I had a rock band, which I was part of. So yeah, music, I'll say.Nic Fillingham:What kind of music did you play? You said a rock band but who would you align yourself with musically?Hardik Suri:So, we were a rock band and my influencers would be your typical classic rock, Led Zeppelin, Deep Purple. In modern rock, I'll say Tool and Dream Theater. Progressive stuff, I like that.Nic Fillingham:What was the name of your band?Hardik Suri:It was called Twisted Flyover. It was named after a flyover. So, when my workers used to come to my place, he had to cross a flyover and that flyover had a lot of circles, so he was kind of confused, there were different exits. So, he always used to take the wrong exit and he just once said, "Man, this flyover is so twisted," and that's how we came up with the name.Nic Fillingham:Nice, and it's not an homage Nic Fillingham:And it's not an homage to Twisted Sister?Hardik Suri:Oh no, it's not.Nic Fillingham:Awesome. Well, Hardik, thank you so much for your time. Thanks for doing great work. We look forward to more updates from you on the security blog in the future.Hardik Suri:Thank you. Thank you for having me. I had a good timeNatalia Godyla:And now let's meet an expert in the Microsoft security team to learn more about the diverse backgrounds and experiences of the humans creating AI and tech at Microsoft.Natalia Godyla:Hi, Karen, welcome to the show.Dr. Karen Lavi:Hey, thanks for having me.Natalia Godyla:Well, we're going to kick it off by just setting the stage a bit. It would be great to hear what's your current role at Microsoft and what does your day to day look like?Dr. Karen Lavi:I am a senior data science lead in Microsoft defender research group. I'm part of the cyber side team, which is cyber security AI, which means that our team is consisting of researchers and data scientists that work trying to tackle problems related to security and protect our customers using the muscle of machine learning and data science.Natalia Godyla:Could you talk a little bit more about the makeup of the teams? How many people are on the team? What kind of backgrounds do they have?Dr. Karen Lavi:So my team is currently five people, including myself. Were coming from completely different backgrounds and actually completely different nationalities as well, which is pretty nice because each one of us brings something very different culturally and technically to the team. We have someone with the background of robotics, to someone that was in the Navy, working on sonars, someone with statistics background and someone that has been for many, many years in Microsoft on different roles. So brings all the aspects of the business into it.Nic Fillingham:And Karen, how about your path to Microsoft? How did you get to Microsoft? What interesting entries would we see on your LinkedIn profile?Dr. Karen Lavi:So I think that you could look at my LinkedIn profile, you wouldn't understand, first of all, what I am, and second where I'm from, because there are so many different entries. So I'm a data scientist, a programmer, a security consultant, a neuroscientists, a medic and coaching girls to code. So I think those are the main things that you would see on my LinkedIn. I joined Microsoft two years ago. Me joining Microsoft was after the returning back to the security field after a few years in academia. Before I was in Microsoft in this role, I was a data scientist in academia, that was after I did my PhD in computation on neuroscience in Switzerland, so moving also States and the countries and roles. Before I was doing my PhD in neuroscience, I was in the security field, I was in day Israeli Defense Forces, I was a pen tester. At that time, I was also doing my BA in psychology. And before that I was volunteering as a medic also during this time. So quite a way until I got here today.Nic Fillingham:That's a fascinating resume. How did you find yourself going from the sort of paramedic world, into psychology, into neuroscience and then here to Microsoft in AI. Was there a catalyst that spurred each of those changes or was it sort of organic?Dr. Karen Lavi:I think the main thing that is associated with all of those different things that I did is my burning need to impact as many people as possible and to help people, and every time it's coming from a different aspect. From the beginning, it was from the medical aspect and then protecting applications against bad actors. And then I wanted to do research and help in combining the medic plus the data, but then academia didn't give me exactly what I wanted because it was indeed helping mankind with progressing science, but I wanted to see the impact of what I'm doing. And that's what I found in Microsoft, which is using my computational skills with protecting our customers against bad actors.Nic Fillingham:You just mentioned taking the sort of medic part and marrying that with data. Can you expand a bit on what that means?Dr. Karen Lavi:Actually, it's very interesting because in what we're doing, and that's the reason that we're using AI and machine learning, is that we're trying to protect in the security world from patient zero. So when someone is getting hit by malware, it's very easy to then, once you know it to block it. But we need to use AI in order to predict it and protect it before we even know that this is malware, to be able to generalize it before we've ever seen it, and to protect the first person that is going to potentially be attacked by this malware.Nic Fillingham:And that's patient zero.Dr. Karen Lavi:And that's our patient zero. So it's like predicting that this disease is going to come or that this disease is going to affect that person before it's actually happening.Dr. Karen Lavi:So besides the fact that being a medic and wanting to protect patients, this is very similar to us, so protecting our customers against malware, there are also some similarities when I was working from the other side of the medic, helping the medics and the firefighters to know to which cases to send ambulances. So there is very limited resources in each city of the ambulances that can be provided to incidents. And when someone is calling 911, that decision of whether to send an ambulance or not, that decision is very crucial because if you're not sending it, then the person might not get the treatment that they need. But if you are sending it to something that they may not have needed it actually for you to come, then you're wasting your resource.Dr. Karen Lavi:In my previous role, in the data science for social good, we built a machine learning model that was trying to predict in real time, whether an incident will require an ambulance or not.Dr. Karen Lavi:So something similar we have done recently in my team where we have for our enterprises, the product that we give them is producing alerts, and they need to respond to those alerts. The security operator is sitting there and seeing go to those alerts. Now, some of those things, we might give them the alert, but it's not as crucial. And if we waste their time looking at it and trying to understand what it is, they might then not invest the time in something else that is more important. We know what is the amount of time that they have and we are trying to prioritize to which alerts they should give the attention to.Natalia Godyla:So what other experiences do you bring from your history to this current role? I know we talked a little bit about your experiences as a medic, but you also have that interesting diversion into computation and neuroscience. So how does that play into your current role today?Dr. Karen Lavi:So besides the fact that my computation on neuroscience, I learned a lot of neural networks and machine learning, which is all of the models that can be transferred and also used in order to classify between malware and clean files, I think that the main thing is that the neuroscience is an interdisciplinary field, and the same is security. Security is a huge umbrella of all those sub topics. And the same way that there is no specialty of neuroscience, each one is coming with a different toolkit that they're trying to investigate a common problem, that's what I do in my team. My team is consisting of different backgrounds, and each one is coming with their specialty. We have someone that is an expert in statistics, someone that is an expert in security, someone that is an expert in reverse engineering, someone that is expert in reinforcement learning. And we're all bringing our toolkits together with us to solve together that big problem that we're facing. That if we will just come with one approach, we might miss all of those other opportunities that we have to solve the problem. But together, it's like a non-linear summation of our powers together.Natalia Godyla:If you are looking at each individual bringing a specific toolkit to the team, do you normally sit back and think, okay, well, I definitely need an expert in, like you said, statistics or an expert in a specific model and then you look to build a team based on filling all of those gaps across the individuals?Dr. Karen Lavi:That's an amazing question because this is actually something that is very dear to my heart. I believe that diversity is not just in the regular way that we define diversity, which is bringing more females, different ethnicities. It's also about the different backgrounds. And the thing that I am looking for the most when I'm looking to add someone new to the team, I sometimes would not know how to define it because the biggest problem is to know what you don't know.Dr. Karen Lavi:So what I'm looking for is someone that is just surprising me, someone that is thinking differently than me. And I'll give you an example. I had interviewed someone and asked them to solve a problem that they give to everyone. And the way that he solved that problem was something that I did not understand so I hired him because if I don't understand it, and that's something that he understands, that's a unique talent that we can bring to our team, a unique approach that we don't have until now. And bringing something that is looking at it from a different point of view, is better than bringing someone that thinks exactly like us. And this is actually a bias that when we are talking about hiring, that one should be really careful from just recruiting mini me's, other people that would do exactly the way what I'm doing, because then we're not going to be able to actually scale and expand. We're just going to solve the exact same way.Natalia Godyla:That's awesome. I love the way you think about innovation consistently, having it on mind. Just sticking with the philosophical level of questioning, when we talk about AI and ML and how you use it in your current role, what does AI and ML mean to you in general, in the big meta sense?Dr. Karen Lavi:The one thing that it's really important for me that whenever I talk to someone that is not an expert in AI and ML, Dr. Karen Lavi:Is to explain that this is not magic. It's not going to solve a problem that is not solvable. But what it can do, it can just take our domain expertise and be able to scale it to a way that a human by itself can not do, and that's the computational power. So I'll give you an example. When we were talking about the anti-Malware product defender, the one that I'm working on, and we want to be able to identify Malware, we need to predict. For me that means to predict when we are now seeing something for the first time if this is going to be a Malware file or not.Dr. Karen Lavi:And AI and ML for me is taking all of our knowledge all over the main expertise from before and all of the samples that we have gathered through the years and understanding whether the attributes that are associating this with Malware and whether the attributes that are not, and building something that is able to learn from that past experience and be able to predict when it first seeing something completely new, if this is Malware or not.Dr. Karen Lavi:If we had someone that had a super mind and remembered everything and was able to access it in femtoseconds, then that person for me would be an AI. But because that ability is not existing for us yet, we have to use computers for it, and that's what Machine Learning and AI is doing for us. It's making us pass that gap that our brain cannot do.Natalia Godyla:Yeah, until we have the magic pill for it, right?Dr. Karen Lavi:Yeah.Nic Fillingham:Karen you mentioned AI and ML and we've used that term as well in sort of posing these questions to you. They're very broad, they're very amorphous. What are some of the techniques that your team utilize or are developing?Nic Fillingham:We hear about Neural Networks and Deep Learning and fuzzing and all these other I think more specific sort of concepts. And they're probably... One's probably a subset of the other, but what do you find are some of the most useful techniques that you find that your team utilize in the work that you do?Dr. Karen Lavi:A lot of the things that we are using because it has to be in an online and it has to be very fast. Currently the computational power is not allowing us to do all of those Deep Learning methods. So when we're talking about those snap decisions, that has to be more of like soft models, like random forests and linear classifiers. This is for all of our online decision making.Dr. Karen Lavi:So, mainly the tools that we're using are those ML classifiers for classification, we're also using a lot of clustering and a lot of unsupervised methods in the backend to understand, for example, that a new version of file Polymorphic Malware, which means that it's like a file that they just change a bit in it. It's still the same Malware, but they just try to trick us. So we're all the time trying to use new techniques and bringing from academia back to our product, new techniques in order, because it is a game that we're playing with the bad actors. They're trying to find new ways to trick us, and we're trying to find new ways to understand that what they're sending is Malware. So we have to innovate and be on top of our game all the time. So the methods are changing all the time.Dr. Karen Lavi:But the one thing that is super important is our ability to understand our data, to see trends, to identify anomalies. That's something that the big data and data science is allowing us and is really important in this case.Nic Fillingham:What are some of the time, sort of constraints that you work with?Nic Fillingham:I mean, so let's say I'm working at my PC and I get an email with a file attached to it and I go to double click it and the Defender service somehow looks at that file and I guess sends some metadata up into the Cloud and it comes back with a determination. I mean, that's happening in a blink of an eye, is it two blinks of an eye? What's the... This is a very short period of time that you're doing a lot of extremely complex stuff. How do you think about that? Are you working in nanoseconds, microseconds, milliseconds?Dr. Karen Lavi:So, that's a great thing. There are a lot of very cutting edge AI and ML technologies that are just currently taking too much time because their hardware is not advanced enough, and we cannot allow ourselves to use it in an online situation because we're talking here about prediction. When you're downloading a file, if it's going to take us now a minute to give you an answer back if this is Malware or benign, you're not going to use our product. It's just going to be too much disturbance to you and it would just not be acceptable and you would rather pay the price of being attacked with a Malware once a year.Dr. Karen Lavi:So our answer has to come really fast and it's a matter of milliseconds for us, which means that we have to make a snap judgment on the client and if we cannot make it, we need to send some of metadata to the Cloud and then bring back the answer because we lock the file at that moment, and you cannot work. So it has to be milliseconds. And that's like again going to the medic like you want to take care of patient zero, but you also don't want to do any harm, and doing harm in this case is distracting the customers.Nic Fillingham:Cool. What are you excited about? What's sort of coming down the pipe that is a tool or a technique or just an advancement in infrastructure that you think is going to allow you and your team to do so much more?Dr. Karen Lavi:I think one thing that we're excited about and we're currently building for our enterprise customers is the ability to help them, not just in the specific protection with the anti-Malware product, but overall in the organization. Learn their organization, use the tools of AI and ML that we know how to use and help them to understand what is needed for their specific org. So, that's something that we are currently working on and I'm very excited about that because I think that a lot of struggles that we've been hearing customers is like, "Awesome. You have this amazing new feature, but how do I know how much impact it will bring? And would it cause any harm to my employees?" And we are able to provide those answers to them and help them to configure it in an automatic way.Dr. Karen Lavi:I think one of the best analogies that we're using now is the self-driving car. We are learning how to drive the car for them and helping them to drive the car. They are now doing it and they're doing it pretty good, but there are sometimes unexpected things. We are able to predict those unexpected things and respond in a faster way because it's a machine and not a human, and we can provide that help to our customers.Natalia Godyla:So Karen, it looks like you've done it all. Are you done with the journey now or is there something after this? What's the next big passion?Dr. Karen Lavi:I think that we're just scratching the surface of what we can do specifically with AI and Machine Learning and Security. There is so much more that we can help our customers and help them to take the wheel away from them and help them to drive the car instead of just giving them the wheel. And that's what I'm excited about for the future, to dive more into that and bring more of those new capabilities to our customers.Natalia Godyla:Is there anything in AI that you're really excited about for the future?Dr. Karen Lavi:Well, there is something that I'm really looking forward that would be developed, which is our ability to build an AI that would replicate ourself. That I would be able to have a lot of mini Karens that would go to all of my meetings and write all the emails that I need to do so I can have time to do other stuff.Natalia Godyla:Time to save the world a little bit more.Nic Fillingham:Karen minions. And they all report back to you at the end of the day with all their progress. All right, well on that note, Dr. Karen Lavi, thank you so much for your time. It'll be great to talk to you again in the future about more things, AI, ML, and Security.Dr. Karen Lavi:Thank you so much for inviting me.Natalia Godyla:Well, we had a great time unlocking insights into Security from research to Artificial intelligence. Keep an eye out for our next episode.Nic Fillingham:And don't forget to tweet us at msftsecurity or email us at with topics you'd like to hear on our future episode. Until then, stay safe.Natalia Godyla:Stay secure. See for privacy and opt-out information.
There are all kinds of powerful features baked into the Windows operating system. One of them is the Antimalware Scan Interface or AMSI. In this episode, hosts Nic Fillingham and Natalia Godyla speak with Ankit Garg and Geoff McDonald of the Microsoft Defender ATP Research Team to learn how AMSI operates, and how they’re utilizing ML to stop attacks. Then they speak with Dr. Josh Neil, a Principal Data Science Manager at Microsoft, about his unique path from music to data security, and why his team is sniffing through weak signals to detect attack patterns.  In This Episode, You Will Learn:  How AMSI protects against threats How machine learning makes it easier to catch attacks The way security experts think about attack methodology How computers can think unlike human brains to solve problems The innovations coming to the world of data science  Some Questions We Ask:  What is AMSI?  How do you differentiate between the benign and malicious?  What’s next for cloud machine learning?  How do define AI  How does music theory impact your work?   Resources Microsoft Security Blog & Geoff’s blog post transcript can be found at Fillingham:Hello, and welcome to Security Unlocked, a new podcast from Microsoft where we unlock insights from the latest in news and research from across Microsoft security engineering and operations teams. I'm Nic Fillingham-Natalia Godyla:And I'm Natalia Godyla. In each episode we'll discuss the latest stories from Microsoft Security, deep dive into the newest threat intel, research and data science.Nic Fillingham:And profile some of the fascinating people working on artificial intelligence in Microsoft Security. If you enjoy the podcast, have a request for a topic you'd like covered, or have some feedback on how we can make the podcast better-Natalia Godyla:Please contact us at or via Microsoft Security on Twitter. We'd love to hear from you.Natalia Godyla:Hi Nic. So we're finished with episode one. We're onto episode two.Nic Fillingham:Yes. Welcome back everybody. This is episode two. We made it. We are now officially expert podcast hosts.Natalia Godyla:Yeah, I got my certification in the mail.Nic Fillingham:Nice. Mine hasn't come through yet. I may have been denied, but I'm glad you got yours. One of us is qualified. That's good.Natalia Godyla:Yeah, this validates the whole thing.Natalia Godyla:So today we have another great lineup of guests. We'll be talking with three experts from Microsoft, kicking it off with Ankit Garg and Geoff McDonald who will be telling us about AMSI and how we're using machine learning to stop active directory attacks.Nic Fillingham:This was a fantastic conversation, and I thought I knew about AMSI and sort of what it did and how it worked. I'm sort of really glad that we sort of asked them to go back to first principles and explain that to us because I really got a much better feel for how the AMSI interface works, and sort of how powerful it is, and what the relationship it is to machine learning. And how Geoff and Ankit and their team utilize machine learning was just fantastic, or fascinating I should say, to hear them talk through it. So it's a great conversation. I hope folks enjoy it.Natalia Godyla:Yeah. And paired with that, we had Dr. Josh Neil, a principal data science manager who talked to us about some really interesting perspectives on AI, which were controversial and definitely enlightening. So ultimately he doesn't like using the word AI and prefers different terminology, and definitely puts it into perspective what we should be using or how we should be defining these terms.Nic Fillingham:Yes, a very compelling argument from Josh on why we probably shouldn't say AI, or perhaps not say AI in the places where we are saying AI, and something I'm definitely going to try and take to heart. I also loved hearing Josh talk about the links between what he does in data science and music as a former professional drummer, which is a bit of a spoiler there from the conversation. But I'm a bit of a musician myself, and so it was great to bump into another musically inclined person in the security space.Natalia Godyla:Yeah. And I am the opposite of musical, so you can carry that piece of the show.Nic Fillingham:But you've got some dance moves, though, so that counts.Natalia Godyla:Yes. Yeah. So to all of our listeners, just know that I have awesome dance moves that I'm doing during the show.Nic Fillingham:When then podcast becomes a video, we'll definitely get that captured at some point. All right. So shall we get on with the show?Natalia Godyla:Yeah, let's do it. Episode two, here we come. Well, welcome to the show Ankit and Geoff. Thank you for joining us today.Geoff McDonald:Thank you.Ankit Garg:Thank you.Geoff McDonald:Excited to be here.Natalia Godyla:We're excited to have you. To kick things off, I'd love to let our audience get to know you a little bit better. So if both of you could share your role at Microsoft and what your day-to-day looks like, that would be great.Ankit Garg:Hey, I'm Ankit. I work in Windows Defender research team in Melbourne. In my day-to-day work we try to analyze the new Decker techniques and campaigns, and then try to think of what will be the best fit to cover those techniques and campaigns using various type of detection approaches which we have. So whenever we got a new technique or a campaign, we try to look at whether we can able to cover it with the client side detection or the cloud side detection, or can we create a new machine learning model to cover this technique at a broader scale? So this is like my day-to-day looks like just trying to discover the malware and all.Natalia Godyla:Are you focused on a specific product for these detections?Ankit Garg:So actually, it's more like not a specific product. We try to look at a broader range of products. So it can be more on, let's say, EXE which is doing something malicious, or a script file which is doing some malicious stuff. But yeah, most of those on the Windows side of things.Natalia Godyla:Interesting, thank you for that. And Geoff?Geoff McDonald:So, I'm Geoff McDonald. I work for Microsoft Defender Antivirus ATP. So I lead a team of data scientists who build machine learning models to protect our customers from malware attacks. So we build machine learning models into the antivirus product itself which run on your device, usually highly performance, low memory overhead, low CPU overhead to not slow down the devices. And then where we build a lot of our machine learning models is for our cloud protection service. So we've got clusters of servers in each region around the world which run real time machine learning models, and that's where we get most of our impact.Geoff McDonald:A lot of what we do on our team involves building machine learning model pipelines. So we'll be coding machine learning and big data pipelines, training the latest machine learning models, and then we'd be setting up pipelines to automatically retrain and redeploy and test these every single day. And we do a lot of machine learning models for a lot of really interesting scenarios. We're talking about our AMSI script behavioral integration capabilities where we've built machine learning models specifically for the scripting engines.Natalia Godyla:So if both of could tell us a little bit more about what you discussed in your recent blog on stopping active directory attacks with AMSI and machine learning, that would be awesome.Ankit Garg:Yeah. So actually in the blog, we try to discuss, firstly, more of a broader machine learning product like the project which we did. So initially we start with a challenging problem like how to detect script-based types of attacks at a bit of generic way. How can we detect those types of attack? As we all know, there's a shift from a normal disk-based attack, which is more focused on p to the script-based attack in last few years. And it is very easy even to obfuscate the script and try to bandaid according to the environments as well. So that is a big challenging problem for us. So what we did is we just try to look at the behavior and the content of the script at that run time using AMSI, and then try to create a machine learning model using that data and try to stop the attacks by looking at the patterns.Geoff McDonald:One of the challenges we often see with our customers is dealing with human operated ransomware attacks. This is a big issue for our enterprise customers where the attackers breach an endpoint on the targeted enterprise. And then they use lateral movement techniques in order to infect the whole network, in order to encrypt everything within the organization at the exact same time, and then demand a very large ransom from the enterprise. So they might ask for, even a million dollars can be a reasonable amount that they would demand after encrypting everything in the organization. So this has been a really big plague upon the enterprise businesses out there. And one of the techniques they use once they breach a box in your network, which is usually through phishing, or maybe it could be a RDP brute force attack, is that what they're going to be doing is trying to infect as many devices on your network as possible.Geoff McDonald:Now active directory is the infrastructure which manages identities within organizations, and it's often the point of target that a lot of the attackers target in order to try to move laterally within the organization, and they're using a lot of these. For example, there are two active directory attacks that our machine learning model stopped in this blog, and both of them were for really different purposes. One is a blue team tool called Bloodhound. Bloodhound is used by defenders in order to help analyze and enumerate active directory within the organization to look at everyone's roles, look at the permissions and access of all resources within the organization. So it's a really useful defender tool. But we actually see this same active directory enumeration tool being used by the attackers to find defense flaws in the organization. So often they're trying to move laterally to the domain controller of the enterprise because once they hit the domain controller, then they have full access to the entire organization, and that's kind of the jackpot where they can encrypt the entire organization at once.Geoff McDonald:Now, the second attack against active directory that was in the blog was the Kerberoasting attack. And it's in order to elevate their privileges within the organization. So they've compromised a single device within the enterprise and they can use Kerberoasting by interacting with the active directory on the domain network that they're interacting with in order to dump the credentials from other devices on other accounts and resources on the enterprise network. So they extract all of these hashes through active directory. Then they can do password cracking offline outside of the target environment in order to crack the passwords for higher privileged users and resources within the organization. Then once they crack those passwords, they can now move laterally within the organization to try to infect more devices within the organization.Nic Fillingham:There's a lot to unpack here in Natalia Godyla:Yeah.Nic Fillingham:... This one. Thank you for that, such a detailed overview to both of you. I'm going to start ... I might even use the blog post sort of as a bit of a treasure map here, and A-M-S-I or AMSI, I wonder if you could sort of give us a description of what is AMSI is that in and of itself, the new technique that you're talking about here in the blog? I don't think it is, but I'd love your kind of clarification of what is it, how long has it been around and what role is it playing in what's being discussed here in the blog.Geoff McDonald:We have a big problem in the security industry where the attackers are using the scripting engines in order to obfuscate impact their script content. So if they have a malicious JavaScript content, for example, they don't just put the malware code directly in the JavaScript. They write their malicious JavaScript content, and then they're going to pack and obfuscate it, so it's really hard to analyze and see the intent of the script. So this is an order to evade antivirus products as well from being able to identify and detect the underlying malicious script.Geoff McDonald:So in order to help with that in Windows 10, we launched a new feature called AMSI, that stands for Anti-malware Scanning Interface. So this is an interface where any application on your computer can ask the default installed antivirus product being used by the user to scan content. So this isn't like Windows Defender specific, this is a Windows feature we introduced, which allows applications to be able to call the default installed AV product, whether it's Defender or Norton or any antivirus product to scan content. It allows us to cut through a lot of the obfuscation and packing that the attackers use to hide the script content and allows us to see the actual intent of the scripts in a more behavior manner, which is a lot more robust to protect the customers.Nic Fillingham:Do I need to do anything, either as an end user or an admin to actually turn this stuff on or configure it, or is it, it's baked in and a part of the product?Geoff McDonald:This is baked in and part of the product. Yeah.Nic Fillingham:That's always the best kind of protection. You don't need to do anything it's already there and it's working.Natalia Godyla:What was most difficult about identifying these types of active directory attacks?Ankit Garg:This is a really interesting question because, okay, so when we actually moved our models to the production, we trying to figure it out what we are ready ... First of all, we are very excited that some of machine learning models, which is stopping risky behavior, move to production, and we are getting the good number of blocks in a particular week. So we are exciting to look at what are all those blocks looking like, what exactly we are blocking and what types of attacks those are. Then we try to dig more into the telemetry of the blocks, and some of the interesting things which come up is the use, like these active directory attacks. So when we tried to look at a telemetry and when we start looking at some of our PowerShell blocks, we start stopping a lot of these active directory based attacks as they are also based on PowerShell.Ankit Garg:And when we try to look specifically what they are doing, we find out that LR model got trained on the behavior we are detecting these behaviors, where some attacker or some pentester try to move literally in the active directory environment, or try to elevate privileges using PowerShell and all. So these are some of the challenges which we have in the past, like to detect these types of attack, which machine learning models is able to overcome or fill that gap.Geoff McDonald:Yeah. One other interesting thing to note is you might be wondering like why machine learning in these cases.Nic Fillingham:Yes, why machine learning?Geoff McDonald:Yeah. So one of the big problems we face is the scale and volume of the diversity of attacks we see in the real world. So we get an enormous number of attacks every single day and to have humans go and analyze and write signatures or write behavior traps for these attacks, doesn't really scale to the scale, the way we needed to, to the volume of attacks we see every day in the wild. So a lot of using machine learning is being able to scale to automatically learn and block these attacks without having to get humans in the loop, which isn't a particularly effective approach to protecting our customers. We're training the machine learning models to broadly defend against these stripping attacks. And these weren't specifically trained in order to detect active directory PowerShell attacks on the network.Geoff McDonald:This machine learning model actually preempted the human signatures, which would have detected and prevent these attacks. So it's learned automatically for us where we don't need to get a human in the loop in order to write more signature based solutions, which we don't rely on, aren't very robust and are lot more reactive approaches. So machine learning is able to be proactive and scale to a way that human response can't.Nic Fillingham:This is what machine learning can do. It's not just about infinite monkeys with infinite typewriters. It's about sort of being able to see the evolutions maybe faster and more efficiently than a bunch of humans sitting in the sock.Geoff McDonald:Yep. Yep. Exactly. This PowerShell AMSI protection was probably the hardest scenario that we were shipping. So a lot of the other scripting engines like JavaScript and visual basic macros, there isn't quite the same diversity of clean, clean scripts in the whole world, as we see with PowerShell. So with PowerShell is just a humongous, enormous amount of clean PowerShell scripts being used by all of these enterprises that are often custom to those enterprises. And it was one of the hardest ones for us to ship. So we had to work through a lot of problems and a bunch of iterations in order to get it successfully working with a very low signal to noise ratio.Nic Fillingham:And as you said, that's because PowerShell is just so prevalent and PowerShell in and of itself is so powerful and also so customizable that there's probably not a lot of overlap between two enterprise customers doing the same thing with PowerShell at same time.Geoff McDonald:Exactly. And like the implications about false positive can be fairly disruptive to an enterprise on top of that too.Nic Fillingham:How do you tackle that problem of trying to sift benign from malicious with something like PowerShell? It sounds like maybe it's an easier, and I use easier in inverted commas that no one can see because it's a podcast, but maybe an easier task, as you say, with some of the other scripting languages, but due to the nature of PowerShell, how do you tackle that?Ankit Garg:So initially when we start looking at the signal to device ratio, we find, as Geoff mentioned, lot of the blocks which our model is doing is very similar to the benign things. So what we did is we tried to narrow down those cases, like why exactly our model is detecting of benign. But when we tried to dig a lot into the data, we try to come up with the new features. And when then we try to look at how can we restrict these benign things from getting detected. And for those, we try to include lot of guard rails, which is more like, "Okay, so we start looking at lot of age and prevalence, and we also start collecting a lot of new features, which we can use." So those features can eliminate these benign samples and able to recognize more of the malicious content.Geoff McDonald:So one of the challenges is because there's an enormous amount of clean PowerShell scripts used custom to each enterprise. So one of the ways that we do to learn that all of that PowerShell content is benign is that we track what we call a healthy machines. So if these are enterprise machines, they're seeing PowerShell AMSI content on their devices, but they've never encountered a threat on the device then it's high likelihood that, that PowerShell content is benign. So actually when we train our cloud classifiers, we're training all of this custom enterprise PowerShell script as benign, and we don't actually have the PowerShell script. We just have a featurized description of the PowerShell script. So we can't actually train on the PowerShell script content itself, but we're just training on the featurized PowerShell script as negative.Geoff McDonald:Now, as Ankit mentioned, it's very hard to get malicious labels of PowerShell on behavior. So one trick that we had in order to improve the quality of our catch rate of true attacks from these attacks is that we look at timelines of devices during a known malware attack on the devices. So if they encountered a malware, we looked at the first time malware was seen on that device in retrospect. And then we look at the PowerShell AMSI buffers from around the time that the malware was first seen on the device and we use that in order to expand our positive label set.Nic Fillingham:That's fascinating, so in order to find malicious, you first focus on benign and clean, and focus your attention there. And then you almost in sort of a post-mortem sense, go backwards in time, looking at telemetry from where there were known attacks and find out what was happening from a feature perspective in PowerShell on those devices. And then you can sort of ascertain maybe where the malicious stuff was happening. Is that accurate?Geoff McDonald:Yeah. That's a great description.Nic Fillingham:That is some fascinating problem solving. You guys must feel pretty good about that one.Geoff McDonald:Yeah. It's really nice to have that shift. It was a challenge in the PowerShell, especially.Natalia Godyla:So what's next for the team then?Ankit Garg:So actually, right now, we are in the process of now shipping the WMI AMSI model in the production, which will include a WMI capability in our suite. And then we are also thinking to work on the .net AMSI as it is pretty new. So we are thinking Ankit Garg:To work on that and also try to shape those models as well to the prediction.Nic Fillingham:And then what's next for the cloud detections or the cloud machine learning that you and your team are working on Geoff, is there anything you can give us a sneak peek on?Geoff McDonald:Oh yeah. I'm really excited. Our cloud machine learning service is really exciting. So we run them all in real time as queries arrive. So this isn't like when your device talks in nearest cluster servers to you, we're running about 90 machine learning models in parallel against every single query, producing classifications using ensembles to make decisions. But the really cool neural part is now in that cloud service in Azure regions which support it, we're getting a GPU inferencing in the cloud. So we're going to be able to scale up a lot of our deep learning models to actually run at the scale we need it to.Geoff McDonald:Because each day, we have about 800 million queries per day, and then we have to run all 90 ML classifiers against each of these in parallel to clump classification decisions. So it's quite large scale problems, but we're really excited about our new GPU capability.Nic Fillingham:So all of those queries are running against physical or virtual CPU's and now they're going to ship over to GPU's?Geoff McDonald:Yep, exactly.Nic Fillingham:Wow.Geoff McDonald:So we're going to be using GPU acceleration for a few of the model types.Nic Fillingham:Wow. That's pretty exciting. Well, we'll have to get you both on the podcast at another time to talk about that.Natalia Godyla:Geoff and Ankit. Thank you so much for joining us. It was a fascinating episode.Geoff McDonald:Thank you so much for having us. It was really a pleasure.Ankit Garg:Yeah, thank you so much for having us. It's a really fun.Natalia Godyla:And now let's meet an expert from the Microsoft security team to learn more about the diverse backgrounds and experiences of the humans creating AI and tech at Microsoft. Today, we have Dr. Josh Neil on the show with us. So Josh, would you mind kicking it off with telling us what is your role at Microsoft and what does your day-to-day look like?Dr. Josh Neil:Sure Natalia, it's nice to meet you. So I'm a principal data science manager, and I work in Microsoft threat protection. We're a research team supporting several products, but focused in enterprise security. In terms of what does my day look like, it's quite busy. That's pretty obvious. And mostly at this point my career consists of mentorship, guidance of research directions, development of research directions and strategy, interface with engineering in order to bridge the gap between research and production solutions for our customers.Dr. Josh Neil:We're certainly motivated to create innovation, but do so in a scaled way that can actually help our customers. We're not a pure research organization. We're very motivated to help protect our customers from threats they face on a day-to-day basis. So there's a real combination between actual research and the scientific method and the needs of scale production computing. And so a lot of my time is spent in understanding the production engineering requirements and letting them know what my team needs in order to bridge the gap between the research and a solution for our customers.Nic Fillingham:How did you find your way into this position? How did you find your way to Microsoft? What was your path to here?Dr. Josh Neil:Boy-Nic Fillingham:We have all the time in the world.Dr. Josh Neil:Okay, great. And I love talking about myself.Nic Fillingham:Well, you're on the right podcast then.Dr. Josh Neil:I am. So yeah, I started out as a music major, a music performance major. I played the drums in high school and was in bands in college. It turns out that a formal education in music wasn't consistent with my passion for music, which was really about performance and playing music, not studying music, if that makes sense. And so after a couple years as a professional musician, I also realized that at large was for me. And so I went back to school, always had another passion for mathematics and computing. Wandered around in various majors, including geology and chemistry.Dr. Josh Neil:But did end up in pure mathematics with a minor in computer science. And then I got a job at Los Alamos National Laboratory. I think I was hired as a scientific programmer. So I was lucky enough to be able to go to school again while employed and ended up getting a master's in electrical engineering and then a master's in statistics and then a PhD in statistics. In those days, and this is in the early 2000s, they didn't call it data science. That's actually a relatively new term. I was a numerical programmer or research programmer for awhile. And then eventually, they called me a statistician. And then it's only recently they started calling me a data scientist. That's okay. I'm happy with that. But the work that I did then, and that I continue to do today is in the application of statistical methods for identification of attacks in computer networks.Nic Fillingham:Staying with data science for a sec, from your perspective, when did the work that you were doing that was initially referred to as statistics or in the statistics realm, when did it start to bleed over into this field that we maybe sort of broadly think of as AI?Dr. Josh Neil:That is a loaded question. Because... And I have these arguments on LinkedIn actually, and you can see my LinkedIn feed for some of this kind of discussion, but I'm demanding that we define AI in the first place. What is it? And people have a lot of different answers for that. I think it's another term which is a bit confusing, just like data science. Because actually under the hood, it's all a bunch of things, and I will be controversial at times about this, but I don't think we have a good concrete definition for that. And therefore, I don't really like to use the term.Nic Fillingham:For all the Dr. Josh Neils' out there, what word should ask laypeople use that is more accurate, if nothing else?Dr. Josh Neil:I guess I've settled on data-driven methods.Nic Fillingham:Data-driven methods?Dr. Josh Neil:Yeah. So they're informed by the data. And the only definition I can really come up with for AI which is appropriate and defensible is when we're trying to actually mimic the human brain intelligence. The neurons that are firing in the brain and the patterns and the learning that we do over time and so forth. Can we write algorithms specifically to try to mimic that? Then I sort of feel like "Okay, that's AI." But actually, we can use computers in ways that brains don't work. And it scales and for problems that humans aren't very good at.Dr. Josh Neil:So should we really be trying to mimic the brain? I don't know, and if we're not, I'm not sure we're talking about artificial intelligence. Although people can argue with me about these, these are just terms, but I think what we're really doing is try to make it as transparent as possible, there's a bit of math, and a bit of computing and a lot of data to try to solve people's problems.Dr. Josh Neil:I could spend quite a bit of time with you talking about explainability. And I know the audience here... I know the feeling among our customers that cybersecurity and AI and cybersecurity has a lot of snake oil in the market. And it's bothered me from the beginning to see intentional or unintentional obfuscation of what we do. Most of the methods that my team develops are focused in explaining what the data is telling us, as well as making decisions with the data.Dr. Josh Neil:So a mistake that some in machine learning make is to focus only on the raw performance of the machine learning model, error, precision and recall type of metrics for their detectors or whatever they're trying to do. I will give up some precision and recall, that is I'll make more false positives and more false negatives, in order to be able to explain what the algorithm is saying about the data. And be able to pass that explanation all the way to the customer. In this case, a SOC. But in the end, we want to be able to give our customers extremely clear answers as to why we think something's unusual, not just "It is unusual and you need to look at it, but why?"Natalia Godyla:What are you passionate about trying to solve within Microsoft?Dr. Josh Neil:Yeah, thanks for asking it. Now you're letting me talk about my passion. That's amazing. So I came to Microsoft in 2018 on purpose, because they were the first company I thought was mature enough in the data collection to accomplish what I'm about to tell you. Okay, so that's a big setup for what I'm going to tell you, which is I believe that signal combination is the wave of the future.Dr. Josh Neil:That no longer should we be focusing on "Oh, that's fish." And "That's a weird login." And "There's malware on that computer." But instead, a sort of comprehensive effort to combine signals across the enterprise in order to identify attacks. Some of the work that we do... A lot of the work that we do is heuristic. So it's a rule that says "If X is less than 17, or Y equals 56 and Z is 37, alert." And it'll alert on very specific behavior. And the parameters there, the 57s and the 36 are actually extremely valuable, because our security experts have worked very hard to Dr. Josh Neil:... get these things right and very precise in identifying attacks. Those are part of the ecosystem. The supervised machine learning to decide malware versus benign or to score malware versus benign probability-wise, and then unsupervised methods, anomaly detection to say, "That was weird with probability X," all these bits and pieces we've been digging so hard into each one of them. We got these massive deep learning models with a billion layers and 4 billion parameters, whatever, to identify malware. Right? We can, "This file is bad." Right?Dr. Josh Neil:I think the, well, the next passion for us, for me and my team, is in the combination of maybe weak signals. "Yeah. You think that's malware, but we can't alert on it because we have too many false positives. This thing is suspicious, but it's not suspicious enough." This is how this stuff gets through. Then the next thing that happens, they disable the security tools on the box, they change the registry so they can survive a restart. But in combination, yeah, we have some suspicion they got in here, but then if we also combine that with some suspicion that they disabled the security tools or they did some reconnaissance, those two together have a really strong multiplicative effect on our probability of detecting true positives and not detecting false positives.Dr. Josh Neil:So the overall performance of our detectors goes way up just by combining these signals together. So a little bit of a sales pitch is Microsoft threat protection is the product to do that, and I'm so excited to be a part of that research team building that product. That's what I'm here to do. I'm just very passionate about that.Nic Fillingham:Could you talk just briefly about the makeup of your team and some of the folks that may also have diverse and maybe unorthodox... although what does unorthodox mean... but paths to Microsoft? What kind of experiences are they bringing? Then what do you also look for when you're hiring new people into your team to do the work that you do?Dr. Josh Neil:Yeah. Great, great question. I work with a lot of students in U Dub and others to tell them that this, too, here's what I'm looking to hire. So the team. Let's see. Some basic tenets are I have diversity in the team, both in backgrounds and... Well, all aspects of diversity. Okay? I very much believe in different backgrounds, experiences, gender, race, ethnicity. I believe very much that those things help us do good research and serve our customers.Dr. Josh Neil:Experience is when it... I don't believe in having a too top heavy team, a principal level, 15 year, 20 years in it. No, I want junior level folks, mid career folks, and senior folks. There's a mentorship pipeline that I like to have where the senior folks get to teach the juniors what they've learned, and the juniors get to learn it. I like that sort of environment of learning and progression.Nic Fillingham:How many former professional drummers are on the team? Just you?Dr. Josh Neil:Oh gosh, is there anybody else? Not on my direct team, but there are many musicians in the larger security research org. In research and in security in general, you tend to find musicians. I like patterns, and so drums and patterns go... Rhythm goes together well. That's appealing to me.Nic Fillingham:Have you found yourself consciously bringing any of musical theory or that sort of pattern creation or recognition through into this work?Dr. Josh Neil:I don't think formally, but I trained from an early time when I was in the elementary school, practice drums, implanting patterns and rhythms into my head. That probably influences what I do. It's not direct, but I certainly have a predilection for identifying patterns and data. That's what I do for a living, and that's also what you do when you're playing drums.Dr. Josh Neil:Although there's this other subtle thing, which is passion. Musical expression, it's magical. I don't get that feeling with anything else. When I'm playing the drums, it's a little bit different than on a whiteboard with a piece of math or a computer.Natalia Godyla:What would you say to students to encourage them to enter a similar space?Dr. Josh Neil:Yeah, thanks, Natalia. I think that we're on the edge of a great innovation time. The data availability, and I've suffered through poor data, but the data has really come along. There's too much of it for us. So the opportunities are tremendous in data science in general, and the combination of data science and security, like we talked about earlier, is extremely nascent. There is much work to be done in a high demand. So I encourage you to study, then work hard and learn how to write code.Dr. Josh Neil:But I think also learn the mathematics and do your homework with the mathematics. Really make sure that you understand the fundamentals of probability and statistics, not just application of black boxes. I tend to hire folks who are builders, not tool users. They're toolmakers, and we really get to the fundamentals and you need to know these. But if you do spend the time, you've got such a tremendous promise in your careers that this old guy would encourage you very much to go with gusto into the future.Nic Fillingham:What gives you hope? It sounds like you're passionate about students and the next generation. Is that the golden, shining light? Is that what gives you hope?Dr. Josh Neil:Yeah, tremendous hope. I've seen so much progress over the last 20 years. It's amazing times to be alive. I think we're miles ahead of where we were in the past, and the future is very promising for defense. We are going to exceed the adversary, and this next generation is the one that's going to do it, I think. So that's what excites me. Don't get me wrong, I'm not quitting today, but I think we'll see this in the next 10, 20 years. It's going to be a good time for security, coming around.Natalia Godyla:Innovation and passion will see us through.Dr. Josh Neil:That's right.Natalia Godyla:Thanks, Joss, for joining us today. Was a great discussion and I loved your definitions or your contrary definitions on AI. It was very eye-opening.Dr. Josh Neil:It was my pleasure, Natalia. Thanks so much.Natalia Godyla:Well, we had a great time unlocking insights into security from research to artificial intelligence. Keep an eye out for our next episode.Nic Fillingham:Don't forget to tweet us @msftsecurity or email us at with topics you'd like to hear on a future episode. Until then, stay safe.Natalia Godyla:Stay secure. See for privacy and opt-out information.
In this episode, hosts Nic Fillingham and Natalia Godyla speak with Arie Agranonik, a Senior Data Scientist in the Microsoft Defender ATP Research team, about building models using deep learning to protect against malicious attacks. It’s complicated work, requiring huge computing power and even larger amounts of data, and it could be the future of threat protection. They also speak with Holly Stewart, a Principal Research Lead at Microsoft, on how building a security team with different perspectives helps to better understand and stop threats. Plus, her journey from the Peace Corps to Microsoft, and how that informs her decision-making.  In This Episode, You Will Learn:  The difference between deep learning, machine learning and AI Why it’s so difficult to program a computer to think like a human How adversarial models learn from each other to prevent attacks Why the best security teams are made up of those with different perspectives How data science can train machines to find things humans were not thinking about  Some Questions We Ask:  What is deep learning?Does a neural network mimic the way the human brain functions?How are behavioral observations evolving to combat sophisticated attacks? How do AI and ML factor into solving complicated security problems?What’s next on the horizon for data science?  Resources Microsoft Security Blog’s blog post transcript can be found at Fillingham:Hello and welcome to Security Unlocked, a new podcast from Microsoft where we unlock insights from the latest in news and research from across Microsoft Security, engineering and operations teams. I'm Nic Fillingham.Natalia Godyla:And I'm Natalia Godyla. In each episode, we'll discuss the latest stories from Microsoft Security, deep dive into the newest threat intel, research and data science.Nic Fillingham:And profile some of the fascinating people working on artificial intelligence in Microsoft Security. If you enjoy the podcast, have a request for a topic you'd like covered, or have some feedback on how we can make the podcast better.Natalia Godyla:Please contact us at or via Microsoft Security on Twitter. We'd love to hear from you.Nic Fillingham:Well hello, Natalia.Natalia Godyla:Hi Nick, how's it going?Nic Fillingham:It's good. I'm in Seattle and surprise, surprise, it is overcast.Natalia Godyla:I'm in Boston and from where I'm sitting, it looks pretty sunny. So feeling good, ready for this podcast. What about you?Nic Fillingham:I'm also excited and ready for this podcast. This is our first one, we're doing it, this is it. This is the first episode.Natalia Godyla:Yeah, and I feel prepared for it, considering all the podcasts I hosted before, which is a whopping zero.Nic Fillingham:Yeah. I also have no experience hosting a podcast, but I've listened to a lot.Natalia Godyla:Which counts. In our podcast we're going to be listening to a bunch of experts. So I think we're primed for that.Nic Fillingham:Listening and asking questions and that's something that I've done over the past 15 years at Microsoft. I've often listened to incredibly smart people, much smarter than me, talk and every now and then I got to ask them a question. And almost always at the end of those conversations, I've thought I should have been recording. That would have been such a fascinating conversation for other people to listen to. And that's what we're doing here on the podcast. So I'm excited that I can bring that to other people through this format. But you're relatively new to Microsoft, so what have you been doing for the past 15 years?Natalia Godyla:Yeah, I'm definitely not a Microsoft old timer like yourself, very much a newbie.Nic Fillingham:Hey, watch it.Natalia Godyla:Well, I've been in the security vendor, compliance vendor space for a while, and I am super excited to just start meeting people within Microsoft. So a little selfishly excited to have the conversations myself. This place is huge, and so each new episode, I get to meet a couple of new people and learn a lot of things.Nic Fillingham:One of the first people we meet in this first interview you're about to hear is Arie Agranonik, who is currently in Israel. And he talked to us from his attic, which he temporarily turned into a mini recording studio, which we very much appreciated. And he's going to introduce us to this concept of deep learning and how that works in a Microsoft 365 Defender, which I think is a great way to kick off the podcast. And then after that.Natalia Godyla:We talked with Holly Stewart, principal research manager, about her path to cyber security and how she builds an awesome security research team. I am particularly excited about this one, she is known as the AI queen and what a legacy that is. To be known within your organization as the AI queen.Nic Fillingham:Absolutely, that's a pretty cool nickname. I would try and get that one on a vanity license plate for my car, if possible. And look, before we jump in with the podcast and the interviews, I just want to say, you've probably heard it in the intro, you may have heard in the trailer, we genuinely, genuinely want this podcast to be your podcast. We want to represent you, dear listener. We want to ask the questions that you want to hear. We want to cover the topics you want to hear covered. So if you have requests on how we can make this better, or stuff you'd like us to cover on a future episode, please, please, please reach out to us through all those various contact methods. And we will do our very best to incorporate it into future episodes of the show.Natalia Godyla:Our goal is to be your voice and to provide a platform for you to learn more from these people.Nic Fillingham:And so with that, let's get on with the podcast.Natalia Godyla:Let's do it.Nic Fillingham:So I'd like to welcome Arie Agranonik, a Senior Data Scientist in the Microsoft Defender ATP research team. Arie, welcome to the Security Unlocked podcast.Arie Agronanik:Hi, thanks for having me.Nic Fillingham:So Arie, can you tell us a little bit about yourself and the work that your team does?Arie Agronanik:So I work in the Defender ATP EDR product, I've been in Microsoft for two and a half years. And in the EDR product, we basically develop models to detect breaches in the operating system. Anything that is attacking the operating system with different attack vectors, we wright machine learning models to defend against that.Nic Fillingham:Now, it is late July in 2020. We are all still in the work from home mandate due to COVID. So that's why we're not in the same room, but Arie, you're actually over on the other side of the globe from us. You're in Israel, is that correct?Arie Agronanik:Yes. I'm actually sitting in my attic in Israel. It's 9:00 PM here. So, yeah, it's near Tel-Aviv, pretty far. And we're still working from home, so lots of fun.Nic Fillingham:Well, thank you for taking the time to talk to us in the podcast. Now you were one of the co-authors of a blog post on July 23rd titled Seeing The Big Picture: Deep Learning-Based Fusion of Behavior Signals for Threat Detection. This was a fascinating blog post.Arie Agronanik:Thank you.Nic Fillingham:I understood several portions of it, but I'm really excited to have you on the podcast so that Natalia and I can better understand what is deep learning and how it's being used. And I wondered if you could give us a bit of a summary of what you talked about in the blog post that was published on July 23.Arie Agronanik:So basically the blog post is talking about a model that we created. We have many models in our product, but this model is a little bit different in the sense that ... I guess in two ways, it's different. First of all, it's a deep learning model, which we can talk about what is deep learning in a minute. And the second thing is, it's using behavioral signals to learn different types of attack vectors that can happen inside of the operating system.Arie Agronanik:And that's very interesting, because if you take it to a higher level, looking at behavioral signals instead of looking at lower level data, is usually what the machine learning models do. You find that those types of models can actually find very interesting attacks that were not seen before, and that's really the goal of building machine learning, or anomaly detection models in security. So that's why I think it's interesting.Nic Fillingham:So let's start with that first term you just use, deep learning. What is deep learning? It's a type of artificial intelligence. Is it a type of machine learning? Can you break that down for us?Arie Agronanik:Sure. So deep learning is basically a neural network with lots of layers. That's why it's called deep. And I think one of the things that made it so interesting in the past few years is three aspects. First is algorithms, new algorithms that were developed to train those models. Second is the amount of data, so we have now big data that we didn't have before. And labeled data, that's very important as well. And the third is compute, so compute is something that, I guess prior to 2005, 2010, we had less of. And also GPU, because deep learning models are based on or trained on GPUs.Arie Agronanik:All those factors together created a pivot point where those types of models became really, really powerful. And the last few years, we've seen many, many breakthroughs in many areas in deep learning. When we think about translation, we think about speech recognition. When we think about natural language processing, understanding language, self-driving cars, all of that, really, is essentially deep neural network that's working behind the scenes to make that happen.Nic Fillingham:When you say a neural network, what is it that defines something as a neural network?Arie Agronanik:First of all, it's an algorithm, it's a training algorithm and there's a representation of the model. The model itself is slightly, you could say, based on how the brain works. So you have neurons which are basically activation functions and those activation functions are only activated when a signal comes in that is high enough. And when you have thousands and millions and billions of those neurons stacked together in the right way, you can create what's called representation learning. Which is basically. If we look at an image, for example, that has thousands and sometimes hundreds of thousands of pixels, and the algorithm can look at the pixels and create layers of intelligence to say, "Okay, what's in the picture?"Arie Agronanik:There's great features that are, as you go up, the layers, those features are saying, "Okay, that's a corner, that's a car, that's a face, that's two people shaking hands," and so on and so forth. And as you go up the layers the network can actually defer or understand better what's going on in those complicated data points or complicated images. So that's just one example. Of course, you can also defer that to any area, like NLP, natural language processing, understanding text, understanding voice and things like that.Nic Fillingham:So is it accurate to say that a neural network is an algorithm that more closely mimics the way the human brain functions?Arie Agronanik:In a way, yes and no. Yes, in the sense that it's neurons, of course, and it's based on the brain, of course. But no, because there's a long way Arie Agronanik:To go before we can reach a point that what's called AGI artificial general intelligence. And there's a lot of hurdles that we need to jump through to get there because the neural network is just a representation, but we still need planning, we still need the memory, we still need the common sense, things like that, that makes our brain in much, much more complicated and smarter than those neural networks. But identifying images, identifying malware, identifying cars on the street, things like that we can already do pretty well.Natalia Godyla:And is that the goal to move closer towards the neural networks acting like the brain?Arie Agronanik:I think it is the goal. I guess there's a lot of research going on in the industry to make it so that we can create AGI. There's lots of companies like DeepMind and OpenAI that actually only do this, only do the research that needs to happen to make AGI. But usually the things that work in the industry are much less than AGI. And they're basically just solving, known problems and trying to solve representation and learning.Natalia Godyla:Right, makes sense. So we already have applications that we can go after. We don't have to be evolved to the state where it's mimicking the brain exactly.Arie Agronanik:Yeah. I think even if we stopped all researching AI right now, and we kind of just start to implement what was research in the past 10 years, it will take us a decade to actually put that in production. So we have a few years to go.Natalia Godyla:And then, what are the challenges that the team had to overcome when building the deep learning models and applying that to the product that we have?Arie Agronanik:The challenge, I think we can classify it into two types of challenges. The first is the data itself and the representation. Our goal was to take a process tree, to look at the behaviors of this process tree, to extract the behaviors that researchers wrote. We call them observations. So observations are basically like you can think of anything that happens in the operating system that could be a little bit malicious, but not distinctively so that they can create an alert. For example, if you create a file in some area on some folder in the operating system where you shouldn't, or you rename a file to a name that is part of like tools in the operating system, that would be considered an observation. So taking those observations and grouping them together into the form that a neural network can access and feeding all that data into a neural network, that's challenge I think, number one.Arie Agronanik:And challenge number two is basically, creating the architecture to be able to learn those features or those representations. And I guess, also challenge number three is to put that into production. I guess, there's a lot of research going on in deep learning, but much less so that things that actually work in production. So, that's the three challenges of thinking this project.Nic Fillingham:Just to clarify, the work that you and your team talked about in this blog post, this is actually in production right now. This is a part of the ATP product and it is protecting customers.Arie Agronanik:Exactly. We have many, many models in the product that they discover those types of things. And this is just one of them, but yes. So it's been in production and is giving a good value.Nic Fillingham:And this model is sort of uniquely qualified at identifying malicious process trees, is that accurate?Arie Agronanik:Yeah. So if you think about what happens when someone takes over your machine, so someone basically sends you a phishing email and you click on some link or open a word document. What happens is actually a script is being executed on the machine. This script might be very, very, thin and very small, and it might just call a network like a C2 or command and control server out there that the attacker is using to download some other file. And that file will also be executed. And then, once those scripts are executed, they actually create a process tree on the machine. So the process tree has many, many processes. Some of those processes do next to nothing, and others might do some malicious activity, but not too much. But if you look at a single process in the process tree, you might not even recognize that it's malicious.Arie Agronanik:So you have to look at the kind of the big picture. And when you look at the entire process tree, you can find that, say the first process went through the network downloaded some. The second process, wrote to the registry. The third process created persistence on the machine, so that next time you reboot the machine, it will actually run itself again. So things like that, if you look at the entire process tree are actually very, very malicious. So what we try to do is to look at all those activities that happen in some timeframe of course, and we try to classify them as malicious or benign. And we have millions and millions of examples. So it's not an easy task.Nic Fillingham:Got it. And so, you talk about in the blog post, the Bondat worm. What was unique about the Bondat worm and I guess, also what was unique about this approach in being able to identify it?Arie Agronanik:So, the Bondat worm I think, was introduced early 2008. What it does is when someone sends you an email or a phishing campaign and you download it and you run it, it will download some more things from the C2. And then it, it might do coin mining or DDoS attacks to other machines, to WordPress sites or other locations. That's what we saw on the internet that happens. But eventually this type of worm, once it's installed on the machine, the attacker can choose to do anything they want, because they have control through the C2. They can send commands and they can do, they have a botnet and the botnet can do really whatever the attacker chooses to do.Natalia Godyla:So when we're thinking about new strains of malware, how are we able to evolve our behavioral observations to combat sophisticated attacks that are alluding our detection mechanisms?Arie Agronanik:Yeah. So if we think about, for example, ransomware, okay? So, ransomware will usually have a very distinctive set of activities that it does. Now, there's different types of ransomware obviously. And each ransomware might have different signatures and different behaviors. But eventually if you look at it from a higher level as a human, you know that ransomware, what it's trying to actually to do, is to scan your machines, scan your computer, and maybe spread itself to other machines. Once it scans and finds a word document or the types of documents, PDFs, whatever, then it goes to each one of them encrypted, save the file, deletes the original file and goes to the next. So that's a pattern that we as humans can really understand. It's quite challenging for machine learning models to do that.Natalia Godyla:That's fascinating when we think about the variation in attacks and how they evolve over time, there's also the variation among different organizations. So, can you speak to the future of customizable machine learning models? For instance, users, endpoints, they all act differently in each organization. So how do you see machine learning models evolve over time to meet that need?Arie Agronanik:In this case, we don't use a pair organization model. We have a general model of all organizations. But what we can do, and that's probably something to think about in the future, is doing transfer learning. Basically, what you do is you teach the model on millions and millions of examples that you collected from many, many organizations, really the same way that they do transfer learning in an image classification. You know in image classification, you can train a model on millions of images. And when you have a small data set of your own with several types of image that we're not trained on, you can do transfer learning. You can freeze the network and only train the last couple of layers on your own data, once it's already learned the millions of examples, and then that network will really be able to classify types of examples that you want. So this same methodology can be used for pair organizational activities.Nic Fillingham:Arie, can you walk us through the process with which you and your colleagues created this technique? How long did it take? Who was involved? And what kind of learnings did you go through along the way?Arie Agronanik:My colleagues that worked on this is [inaudible 00:19:15] and [inaudible 00:09:16], and both of them may helped a lot with the blog. It took us a few months to really get the collection process working correctly. When you train a neural network, when you work on a neural network, part of the challenge is to find the correct architecture that will work best, given the data that you have. So that process also takes a while and you have to do many, many experiments on different architectures and different types of parameters, what we call grid search or parameter sweeps, until you find the right architecture. And then once you do that, once you found it, now it's ready for testing and production. So, that took us a while Arie Agronanik:... to get there and eventually we got there.Nic Fillingham:What kind of tools do you use to build a model like this, are you in Python? Are you in Kusto?Arie Agronanik:To extract the data, we use Azure ML and we use Cosmos as well. But to train the actual network, we use Python and different tools and different libraries in Python ecosystem for AI like Keras, TensorFlow, PyTorch and these types of libraries, they are very popular these days.Nic Fillingham:I think my final question is what do you know of the inverse of the work that you're doing? So the bad actors, the folks that are out there creating this malware, it's getting more and more complex every day, are they utilizing any adversarial machine learning or how are they deconstructing the work that you do to try and create more complex malware?Arie Agronanik:I think the advantages of our product in general is that all the models are in the cloud. So the attackers don't have the luxury or the capability to actually look at the model and start dissecting or reverse engineering it. So that's something that we saw in different areas where if you give the model, if you show them the model, they can actually start running it against many, many examples, and then start doing adversarial machine learning. But on the flip side, we also always try to develop models that can be given adversarial examples and they'll try to detect them as well, and that's a very complicated process because what you have to do is you have to basically train two types of models. One, the attacker model and one is the defender model, and those two models are basically learning from each other. One model might find different variations of the same data point to attack, and the other model will learn from that data point, how to learn that it's actually fake or how to learn that it's malicious. So that's something I feel that we're investing in as well.Natalia Godyla:One last big question. What's next for deep learning.Arie Agronanik:Oh, wow. So I think there's many, many areas that we can go into, anything that's got to do with representation learning. I think in security in general, we're a little bit late with deep learning because I think it's very difficult to take the data and to make it so that it will be able to be represented to the deep learning model. This is a constant challenge in security. When you think about images or sound or any of those other areas, the data is continuous and it's much more fitting to a neural network to learn from. So I think one of the challenges will be to create more and more types of models like this that can look at the security situation or security data and be able to learn from it given millions or even billions of examples. That's I think one of the challenges.Arie Agronanik:And again, adversarial machine learning is also a very big challenge. I think a lot of people are working on it, and in general, putting those models into production is also a challenge because they usually require a lot of compute. If you think about like a normal or standard linear model that we used to work on like for 15 years, it might have, I don't know, 10,000 parameters or 5,000 parameters or whatever, and a deep learning model might have millions or billions of parameters. So you have to have really good hardware to actually run it at scale and this is, I think a big challenge as well.Nic Fillingham:So Ali if listeners of the podcast would like to learn more about deep learning and the technique that you and your team have created. Is there some way that you recommend they go to read more either about this directly or related topics?Arie Agronanik:If you look at our blog posts, there's links to other machine learning articles that our team already wrote, specifically on PowerShell, and there's a lot of things on the Microsoft Security blog as well that relate to machine learning.Natalia Godyla:Thank you Ali for your time and all of the insights that you shared today. It was a fascinating discussion, and definitely one I'm going to keep diving into.Arie Agronanik:Thank you for inviting me. That was great.Natalia Godyla:And now let's meet an expert in the Microsoft Security team. So more about the diverse backgrounds and experiences of the humans creating AI and tech at Microsoft. Today, we're talking to Holly Stewart, Principal Research Manager in the Microsoft Offender Research Group. Welcome Holly. Thanks for joining us.Holly Stewart :Hello. Thank you for having me.Nic Fillingham:Awesome. So let's start with, if you could just give us your title at Microsoft, but maybe more interestingly, walk us through what the day-to-day function is of your role.Holly Stewart :Sure. So I am a principal research lead at Microsoft and I work in the endpoint protection side of research. I like to say our team's superpower is using AI to help protect people. Machine learning and data science techniques are used everywhere within our research team, but with our team we have a primary focus on using those techniques to try to help people and keep them safe.Nic Fillingham:That's awesome. And you run a team, is that right Holly? How big is the team?Holly Stewart :It's about 25 now.Nic Fillingham:Yep, and they're all in the AI data science realm.Holly Stewart :Yeah. Actually, they're this super interesting mix of researchers and data scientists, and they come from all walks of life. We have folks who are security experts who really understand what threats do, how they work. Some of them understand criminal undergrounds and other things like that, and then we have data scientists that come from many different facets. many of them, not particular experienced in security, but some may be an expert in deep learning, another person may be more on anomaly detection side. But you take all these folks with different perspectives and different strengths and you put them together and really cool things happen.Natalia Godyla:I love that. And speaking of backgrounds, what was your path to Microsoft?Arie Agronanik:My path to Microsoft was I'd say a little unconventional. I studied International Business and French in school. I thought I would end up in the Peace Corps in Africa somewhere, and instead I ended up working at a security startup out of Atlanta, Georgia for many years and found my love and passion for security and data science. I've worked with a ton of researchers in my time and really found that data science was the way forward for me. It was the way of the future for me. So we got to Microsoft where we have this amazing data, amazing researchers, great compute power from Azure, and it was my perfect world where I could take all of these ideas about how we can use data science to solve customer security problems, and really put that into practice here.Nic Fillingham:So, Holly, you talked about learning French and what you studied at college. What other things in your education, your history pre Microsoft do you feel brought you to where you are now and that you were using in your day? Perhaps things that maybe seem a little unorthodox?Holly Stewart :You know, I'll say that I grew up with a really strong work ethic. My family actually comes from farming and my father has this really strong work ethic. He gets his guilt complexes about, if he's not doing something productive, he hasn't made the... his day is not complete, and somehow I am instilled with that. So when I got into security, I kept seeing so many problems. You're the threat de jure. Every single day, we're just bombarded with information. It's an overload, and I always thought, how can we better solve this problem? How can we help people really understand what matters? And when I started getting into data science, I thought, this is the way, this is how we can make better decisions, help people make better decisions and help protect them in a way where... focusing on the problem de jure really wasn't getting us anywhere, really wasn't moving the needle.Nic Fillingham:So perhaps that drive that maybe thought you were going to the Peace Corps, you're utilizing a similar motivation there, but now in the data science realm.Holly Stewart :Yeah, absolutely. I mean, I love being able to say that I go to work and the work that my team does, we are trying to help people every single day to keep them safe, keep them protected. It's something that I feel good about.Natalia Godyla:That's great, and how does AI and ML factor into that when you're thinking about all of these big complex problems you want to take on?Holly Stewart :Yeah. It's a great question. Like if you think about how maybe we traditionally approached security research where a researcher might reverse engineer some malicious program, figure out what it does, find some heuristic techniques to be able to detect that in the future, make sure those heuristic techniques don't detect the good things that we want our computers to run. That takes a lot of time, and the truth is that now where it has become so complex, that there's literally hundreds of millions of features that feed into what makes malware malware. It's really difficult Holly Stewart :The human brain to wrap your mind around all these permutations, but that's the beauty of Machine Learning and AI, it's built for that. And so we take this incredible ecosystem diversity from benign applications to malicious applications, we feed that information into the machine learning systems. We train them how to recognize good from bad, and they can come up with these permutations that the human brain just wouldn't be able to wrap their heads around. And that's really how I connect all of those things together in our day to day.Nic Fillingham:Got it. And so what types of... When we say AI and ML, that's a relatively broad set of acronyms there. What type of techniques, what type of approaches do you and your team use, or where are you sort of heavily invested?Holly Stewart :We invest in lots of things. So if I break down and I'll say "AI" in quotes, because I kind of use it interchangeably to really just mean data science and data science approach. We use many different techniques from what you call supervised machine learning to unsupervised machine learning. With supervised machine learning you're using signals to help teach the machine how to deck something new. So I may take a set of saying 100 files and 10 of them are bad and 90 of them are good. I extract a bunch of features from those files and then I feed that into machine learning system to teach it how to detect new things that are similar to those files in the future. So, that's what you call supervised.Holly Stewart :Unsupervised is really good at finding what we call the unknown unknown. So in supervised learning you're teaching it's something that you already know, and it just gets better at that. With unsupervised you're trying to find those pockets of uncertainty that maybe haven't even been classified before, or maybe should be clustered together, or perhaps in using past data you find that, hey, this is an anomaly something I haven't seen before that doesn't have a label, but that could indicate that something bad is going on. And so we really use a combination of all of these approaches to help train machines to amplify human knowledge, and also find the things that maybe as humans we were not thinking about in the first place.Natalia Godyla:Can you share a couple examples of how this AI and ML is driving some of the Microsoft products, even products that like Nick said we use day-to-day?Holly Stewart :Yeah, absolutely. So there are a lot of files that use what we call social engineering to try to trick people into opening them. So, one example that we saw over the past year is these attackers were using local business names and making look like they were sending an invoice from a local business name. I think it was a landscaping firm or something like that and so they were using that invoice that looked like it was from a local landscaper, sending it to these other businesses to try to trick them into opening up this invoice. Since inside it, it led to this phising site and then they'd try and collect their credentials. And so when you're just looking at this file, you may not see that it looks benign, but the machine learning system because it was able to extract all these different features from that file, it was able to see, hey, this is not a normal type of invoice that I would see from a legitimate business and it was able to flag that as malicious and help keep those customers protected.Natalia Godyla:So, Holly, what's next on the horizon? What are you most passionate about trying to solve next?Holly Stewart :Sure. So today we've done a pretty good job of using AI to help discriminate malicious software from benign software. It's not perfect, but we've made a lot of progress in that area, but what's next on the horizon for us is really deeper than that. So it's great to discriminate malicious from bad but what more can I learn from that? Say, for example, if we understand the entire kill chain of that malicious activity, from how it arrived to the victim to what it did after, if the victim installed it or clicked it to the final sort of motive of the attacker.Holly Stewart :And if we can understand that entire story, we can look at all of the pieces in that, what we call kill chain and be able to provide protective guidance and automate protections to essentially learn from what attackers are doing today and make our defenses stronger and stronger over time. And that's really the evolution of AI in security is to help automate that for the customer. Because the amount of threats that we're facing, the amount of security information is an overload and we have to get better, we have to automate, and we have to use AI to do it, to really get to where we need to go.Natalia Godyla:And how far away do you think this next step in the evolution is?Holly Stewart :I'm sure I'll be working on it for the rest of my life.Nic Fillingham:Holly, do you have a Twitter account? Do you have a blog? Do you have anything you want to promote? If folks want to learn more about you, your team, if you're hiring.Holly Stewart :So we post all of our content on the Microsoft Security blog so you can find it there. And we are hiring data scientists here in the next week or so we should have the postings up.Nic Fillingham:Great. And so you would find them on the Microsoft Careers website probably under data science?Holly Stewart :Under data science or look for Defender and data science and you'll find us.Natalia Godyla:Thank you, Holly for your time today it was fantastic to hear about your insights on AI.Nic Fillingham:Yeah. Thank you, Holly I know you're busy, you're running a big team doing some great work. We really appreciate you coming on the podcast.Holly Stewart :Thank you.Natalia Godyla:Well, we had a great time unlocking insights into security from research to artificial intelligence. Keep an eye out for our next episode.Nic Fillingham:And don't forget to tweet us @msftsecurity or email us at with topics you'd like to hear on a future episode. Until then stay safe.Natalia Godyla:Stay secure. See for privacy and opt-out information.
Security Unlocked explores the technology and people powering Microsoft's Security solutions. In each episode, Microsoft Security evangelists Nic Fillingham and Natalia Godyla take a closer look at the latest innovations in threat intelligence, security research, and data science, with a special focus on demystifying artificial intelligence and machine learning. Be sure to listen in and subscribe!Transcript(Full transcript can be found at Fillingham:Neural networks, clustering, fuzzy logic, heuristics, random forests, unsupervised learning. These are just some of the concepts being researched and utilized in security today. Natalia Godyla:But what do they mean? How did they work? And who are the people that create them? These are the questions and more we'll explore in Security Unlocked, a new podcast from Microsoft. Nic Fillingham:Hi, I'm Nic Fillingham.Natalia Godyla:And I'm Natalia Godyla. In each episode of Security Unlocked, we'll deep dive into the newest security research, threat intel and data science, with a special focus on demystifying artificial intelligence and machine learning. Holly Stewart:We use many different techniques from what you call supervised machine learning, to unsupervised machine learning. Unsupervised is really good at finding what we call the unknown unknowns. We really use a combination of all of these approaches to help train machines to amplify human knowledge and also find the things that maybe as humans we were not thinking about in the first place. Nic Fillingham:And we'll profile the diverse and fascinating people working on security engineering, research, and operations at Microsoft to learn how they made their way into security. Dr. Anna Bertiger:So I find villains in computer network. It's all the benefits of a job as a superhero with none of the risks. And I do that using a combination of security expertise and mathematics and statistics.Nic Fillingham:And what surprising aspects from their history play a role in how they tackle the biggest challenges in InfoSec.Dr. Josh Neil:In research and in security in general you tend to find musicians. I like patterns. And so drums and patterns, rhythm, goes together well.Dr. Karen Lavi:Before I was in Microsoft in this role, I was a data scientist in academia. That was after I get my PhD in complication on neuroscience in Switzerland. So moving also states and countries and roles. Natalia Godyla:If like us, you wonder about the ever increasing use of AI and ML in the security space and want to know more about how it all works, then this is the podcast for you. Nic Fillingham:And if you have a topic you'd like us to unlock on a future episode, please contact us at or via Microsoft Security on Twitter. We'd love to hear from you.Natalia Godyla:Security Unlocked launches on all major podcast platforms October 14th, 2020. Be sure to listen in and subscribe. See for privacy and opt-out information.
Download from Google Play
Download from App Store