Defeating Deception: Bayesian Analysis

hello again everyone and welcome back to the underground so when it comes to the world of intelligence analysis and statistics and spies and Espionage and all the cool stuff like that as we all know there is a lot of nerdy stuff that goes on behind the scenes that really doesn't get shown the light of day and among that nerdy stuff one of the things that you'd be hardpressed to find as a harder to explain topic is the idea of basian analysis so I thought that by covering a topic that is exceptionally hard to to grasp we can really see that it's not that hard to understand and it's quite helpful in very specific situations so what is Bay analysis well Bay analysis is really any kind of analysis that is using bay theorem or Baye equation if you want to call it that which is a way in statistics of determining the likelihood of something based on followon information so again let's not get super wrapped around the axle when it comes to the mathematics of this so in the real world base equation allows us to have an initial hypothesis of probability like say we say that there's a 10% chance of something happening well we can have that assessment right up front but if we get new information how does that assessment change that's really all that this is it seems like it's more complicated than that but that's all this does it allows us to change change our initial assessment based on new information so again forgive me if some of the words are not correct in this but the reason that we have to kind of simplify this a little bit is because of this this is Bay's theorem looks pretty scary doesn't it so uh this idea is kind of hard to explain but I promise you as scary as this equation looks it is very very simple uh it's almost too simple so let's bring up our main graphic here and let's work through Bas theorem first of all we have to start with a hypothesis right so for the purposes of examples today let's hypothesize that there is a 10% chance that China will invade Taiwan within this year okay let's start with that hypothesis I'm not saying that's an accurate hypothesis but we've got to start from somewhere and I thought that by using a more real world example this might actually help us out quite a bit so let's start with the hypoth IIs let's just say that you know I'm an intelligence analyst and my job is to figure out you know what's the Chan my boss asked me you know hey what do you think about China invading Taiwan this year are they finally going to do it this year well I can say well I think there's probably a 10% chance of that happening this year so let's go ahead and put that on our our little chart here just to help kind of show this visually let's say that the Yellow Block represents my assessment which is 10% yes 90% no and just to simplify this even further let's say that we can think of this in terms of uh Invasion yes or Invasion no right and if you want just to help you kind of break things out a little bit more mentally you can draw an imaginary line uh kind of dividing up these ideas because we're going to be picking apart the yes side of this uh little chart here and we're going to also be breaking apart the no side so again just saying 10% chance of Taiwan being invaded 90% chance that's not going to happen okay now imagine for a moment that your boss comes in uh he comes into your office and says hey look at this new Intel report we got all of these Chinese aircraft all all up and down the Taiwan straight we've got all these major indicators of an imminent Invasion well you can sit back in your chair and say all right um well that's cool um but my assessment doesn't really change that much and your boss may ask you well why is that doesn't it look like we're getting ready to be you know involved in a major conflict here well you can sit back and say no and here's why so bringing our chart up here let's look at the yes side of the equation let's say that we're correct right let's say that uh there's a 10% chance of China invading Taiwan and based on this new information we have to ask ourselves if China were to invade Taiwan what is the percentage chance that we would see this Behavior right the increased Aviation activity well just me thinking I would assign a pretty high value to that because that's why my boss is in my office right he's trying to say hey look this is a major indicator of a conflict happening so if China was going to invade Taiwan if an invasion was yes then we would see this Behavior maybe 80% % of the time can't really ever be sure right China may want to do a sneak attack or something like that so let's just fill this in on our chart here in green and let's call this the likelihood now for those of you who think oh my gosh this is still this is confusing here is the next part which really is the entire Crux of the whole thing and that is let's move over to the no side of the chart and we have to ask ourselves all right how many times or what's the chance let's just say what's the chance of us seeing this activity this increased you know aircraft activity what's the chance of us seeing that even if China is not planning to invade right so if China is not planning to invade at all what's the chance we're going to see this Aviation activity anyway right no matter what so on this chart here let's go ahead and break break that out so I'm going to say just as my own assessment that even if China is not planning to invade let's say they're not planning to in invade well 40% of the time we're going to see this Aviation activity anyway because again Aviation activity is something that China is going to be doing pretty much all the time just to one collect intelligence but also two to keep Taiwan on their toes so we're going to see this Aviation activity kind of a lot right maybe not more than half but let's just say for the sake of argument we're going to see China conducting Aviation activity 40% of the time if China has no aspirations of conducting an invasion if invasion is no then still we're still going to see 40% of the time China conducting Aviation activity which the inverse of that means that their chance that they're not going to do anything at all and they're not going to is a 60 60% right I know that these percentages are probably the wrong way to explain this but it all comes together right now so now that we've plugged in our variables here and we've kind of assessed you know what the chances are of things happening let's bring back in our chart our uh equation our terminology and let's put all of this together and match everything up so that we're all good to go on how this works so starting with with the color yellow if we look back uh that was our initial assessment right we're going to call that the prior this is what we started with that 10% chance right there's a 10% chance that Taiwan is going to be invaded by China this year so we have to remember that magic number and in this case it's 10% uh we're going to convert that to a decimal so 0.1 now we have our variable introduced whatever that is it doesn't really matter but we need to think all right what is the likelihood that we would see this variable occurring if the answer is yes so in this case we're going to say that well if China was going to invade Taiwan there's an 80% chance that we're going to see Aviation activity so that's where we get that 80% from so we're going to express that as the color green again we're going to convert that to a decimal now moving over to the other side the no side of the equation because this is where it all kind of comes together if you just do the yes side of the equation the yes side of our little example here you're going to probably over assess uh the chances of this happening so and remember our our prior means that there's a 90% chance of no Invasion happening if China is not planning to invade at all what's the chance that they would have increased Aviation activity and I'm going to say they're going to do it 40% of the time right they're going to be doing increased Aviation activity 40% of the time even if there's no Invasion well that conversely means that that 60% has to go somewhere so 60% of the time that there is no Invasion there's going to also be no Aviation activity again we're going to convert those two percentages to decimals and we're going to plug it into our equation here from here it's just basic math and if we reduce all of these down we actually get an answer of roughly .25 now what does this number mean this is our answer this means that we initially assessed that there is a 10% chance that China will invade Taiwan well when we add in this new variable this now means that there is a 25% chance that China will invade Taiwan on now if you had looked at this from the very beginning and not gone through this crazy math stuff you probably would have said oh it's it's a lot higher than that which is why your boss is standing in your office your boss is standing in your office saying good grief man there's there's aircraft in the skies right now and you can say well yeah but still considering all the factors the math is not adding up and yeah it's much more likely you know 25% % chance is higher than a 10% chance but it's not a 95% chance is it so really this is all the what the Baye equation is for it is for really conducting a sanity check to make sure that you're not overreacting to data as it comes in you don't want to get a new variable in and then just go Whole Hog on it and change your whole assessment based off of one new variable that's all this is so right there's nothing more to it than that and this kind of once you kind of do this a few times like pick an example and work through uh the the way that I have this equation laid out for you you may have to take some screenshots or take some notes or whatever to work through this a couple of times with your own like examples and you will see that really this is not that complicated and it makes an awful lot of sense and this is where some of the criticisms of the whole basian analysis uh thing can come in because it sounds really intense right it sounds really like high level statistics um you know and it may be if you get way deep into it but when it comes to like the social sciences or you know intelligence analysis really when the rubber meets the road it doesn't provide that much value uh it does help and really you don't necessarily have to go through this equation to kind of figure out some of the diff disadvantages the first disadvantage is that this whole system relies on your initial assessment being decently correct uh you really have to have somewhere to work from and if your initial assessment is really off base then you're going to find that the statistical analysis really is just kind of reinforcing a wrong idea so you may find that you know you have a 10% chance of something happening well now it jumps up to like 95% chance of happening and you may say well this is this is really off base what did this new data do to change my equation so much well it didn't really it's just that you're wrong right to start with another disadvantage to this is that it kind of you know really relies on your assessment of that second variable and if that analysis is wrong then you know you're going to be pretty inaccurate a lot of the time a third problem with this is that it oversimplifies what is sometimes a very complicated set of variables for instance um there are ways of computing more than one variable uh but really in the real world you're only going to be you're only going to be dealing with the one so you could just be doing all this mathematical effort really for something that's just kind of Common Sense and that kind of Lies with one of the final disadvantages is that and this is where a lot of the critics of uh basian analysis kind of come in is that this really isn't that useful for like social sciences or um things where we already have to kind of guess the probability anyway so you'll find a lot of times that people tend to you know lean on basian uh analytics really just as a way of making themselves sound smarter than they really are because all of this can be boiled down to one really simple idea and that is that if I have evidence that supports my idea and that evidence also doesn't support my idea then that evidence doesn't really mean anything you know that's kind of basy in thinking in a nutshell is you have to realize all right does this evidence support my idea or does it not or does it do both does it both prove and disprove my data if when we add in that evidence that variable to our line of thinking and it doesn't change either way if you're right or if you're wrong then that evidence didn't really do anything it if you think of it kind of makes sense you know going back to our example well you know we think that you know China May invade Taiwan 10% chance all right but even when they're not planning to invade they still do this variable 40% of the time so that really kind of shows that yeah they're you know seeing this activity is going to be an indicator of an invasion but how strong of an indicator well 25% is what it turns out to to be using our our example so I know this is kind of complicated to think about but this may help you uh perform kind of like a sanity check a lot of the time if you're trying to figure out well I've got this new information here how does this really affect my assessment at the end of the day how does this really change my line of thinking and does this change my hypothesis any uh sometimes it may go up it may go down uh depending on you know what you think is likely to happen so again very very complicated topic we may talk about this more in the future but this is just kind of getting us started by understanding what basian analytics is and how we can use this to affect our assessments so thank you all for watching give it a try and see if it works out for you and we will see you next time and as always fight in the shade