Analysis of Competing Hypotheses

hello again everyone and welcome back to the underground in a fifth generation war or a propaganda war or whatever you want to call it there's a lot of fake information flying around out there in the world today and it makes it really hard to figure out who actually did what if some incident occurs a lot of times people are really confused and trying to figure out like when an incident occurs the incident itself may be quite clear but who actually did it is a little bit more debatable one great example of this is the recent AT&T service outage that occurred back on February 22nd for those of you maybe who were unaffected by this you probably heard about it anyway uh this was the case where AT&T's service for the majority of AT&T customers Nationwide went offline at the drop of a hat and it stayed offline for a very long period of time now again in our crazy information space the theor started flying did they not uh we had every kind of possible Theory as to why the this uh system went offline uh and most importantly who was the culprit right in a situation like this where we have several different competing ideas for uh the reason behind something happening a good way to analyze this in a more professional and doctrinal way maybe a good way to look at this is through the lens of the analysis of competing hypotheses this is a structured analytic technique that we can take advantage of to maybe help guide our thoughts in the right direction this is again one of those kind of nerdy Intel tactics and techniques that we can lean on to maybe make sure that we're looking at things from the right perspective and in today's world I think it has a lot of value for the average person out there too so what is the analysis of competing hypotheses well really it's just a very simple way of comparing ideas as different facts and evidence comes out well really it's easy just to start with the chart so this chart here is usually the main tool and the main way that analysis of competing hypotheses or AC is carried out for those of you who are wanting to know kind of a little bit more Doctrine about this this is what we call a matrix uh some of you might see that this is a little bit familiar to the Carver technique which is again another type of Matrix again it's just Excel for you know most of us right it's just a simple table that you can make it either Excel or in PowerPoint whatever you want to do you can write it on the of a bar napkin it doesn't really matter uh but this is what the chart looks like and this is usually how it's laid out so you have different columns that show your different hypotheses so using the AT&T service outage example let's take apart some of the most common hypotheses that there are regarding that incident so just going to stick with three main ones for now uh the first one is that it was a Cyber attack and China did it right uh that's the most common theory that people seem to be going with when it comes to anything to to do with critical infrastructure and things like that the second theory is that it was also a Cyber attack but the American government did it again you know false Flags again something people might not want to think about but again we have to examine all different possible factors and really this is a pretty likely uh hypothesis so again we want to compare it and throw it in there right when we're doing these analytical things we have to consider things that maybe are not politically correct a lot of the time and and uh even if it doesn't seem likely even if it's not uh something that we want to talk about it may be socially challenging to talk about in an open space a lot of the time we still have to consider it right so just going to say Hey you know Cyber attack but it came uh as part of a false flag operation that's a hypothesis right and the final hypothesis which again some people may want to believe just outright without doing the analysis is that it was an accident just simple either an incomp ENT worker or some kind of software glitch or a software problem of some kind but nothing the fairy it's just a pure accident so these are the three main hypotheses that we're going to be comparing with each other and comparing against the facts of what happened so looking at each row uh we can see in that left on the left hand side of the Matrix here we have different rows that can express different facts of the case this is where our evidence comes in and I'll just go ahead and drop in a few different facts of the case the first one being that the incident lasted most of the day now that's just a plain fact right it doesn't support one thing or the other it's just a fact of what happened we'll get to the analysis bit in just a moment the next fact that we have is that most AT&T customers were affected so again just a blanket fact and then finally we have Verizon customers were largely not affected again there were Verizon customers that were affected but we're going to have to generalize to some degree when we're working through these problems so we have our chart filled out right we have our hypothesis along the top and we have our facts of the case along the side now before we get into this let's take a break for a moment and let's try to figure out some of the problems with the system as we go along I think that'll be a good challenging experience right so as I go along and explain how this process works and how most people conduct it try to take some some notes and try to find the points in which personal bias comes into the equation right let's try to pick apart this process and see does it really do what it's supposed to do after all examining uh any kind of analytical process whether it be you know this a or the Carver technique or something else when we're using a structured analytical technique what we're trying to do is remove personal bias so let's keep keep an eye out for that right go ahead and take some notes just kind of mentally remember some of the points along the way as I explain this where personal bias comes into the equation and where personal bias can skew the results I'm showing you right now the way the most people tend to do this but there is an alternative way that allows you to take some of these biases out of the equation it allow us to be a little bit more objective and it allows us to get a little bit more uh accurate results now once we have our chart laid out here our whole goal for this is to go block by block and determine whether or not each of these factors these bits of evidence apply or are compatible or consistent with the hypothesis now there's many different ways of of thinking about this and notating this a lot of times if you're if you're going to see this technique in other like Publications and manuals you will some sometimes see this expressed as like a plus or a minus sign meaning that yes this piece of evidence is consistent with our hypothesis or no this piece of evidence doesn't really match our hypothesis sometimes you'll also see it as C and I consistent or inconsistent with our evidence this may be a little bit hard to explain so let's go through and work through each one of these and I'll show you what this actually means I prefer to use yes and no sometimes maybe depending on what you're trying to do but let's just stick with yes and no compatible or incompatible for each hypothesis so let's start with the first one the incident lasted most of the day now this is kind of a weak fact but it is a fact so we have to start with it now if China did it if China did it would the would the incident have lasted all day long I think so yes because if it's a major nation state Cyber attack they're going to want you know maintain the most Effectiveness so right up front we're going to start with a yes in that block moving on over false flag situation the more nefarious one right if the United States did it if some federal agency was trying to either make a point or you know have some kind of political incentive to do it would the attack have lasted all day me personally I think so yes because both United States and China have similar cyber capabilities we're talking you know Nations state level stuff so they probably have the ability to do that so we're going to say yes for that block now coming to the last block here if there was an accident did it would it have lasted all day long statistically speaking no right this is kind of a challenging one but if there was an accident then most of the time accidents are resolved quite quickly even software glitches they don't tend to last a whole day that maybe few hours maybe a short just a few minutes but it's very atypical for a nationwide service outage to last all day long so for this block we're going to say no this Factor does not jive with the whole accident story so again hint hint maybe this is a way of of uh injecting a little bit of bias here and there so we'll come back to that in a moment though let's move on down to the next one this one's a little bit more clear most AT&T customers were affected for this one I'm going to say yeah this is likely for all three if China launched a Cyber attack most people would probably be affected if the United States you know a federal agency launched a Cyber attack then yeah it would probably affect most people because they have similar cyber capabilities and finally if it was an accident it does have the potential to affect most people but now coming down to our our last Factor our last little bit of evidence that we have on our chart here Verizon was not affected isn't that quite interesting so in this case if China did it if China wanted to launch a Cyber attack I would think that Verizon would be affected you would see outages along multiple cellular uh networks and Verizon included Verizon not being affected that's inconsistent with uh China wanting to do it so we have to say no here same thing with a false flag if there was a false flag incident then you know maybe more than one network would be affected however going back to our accidents story that does match up with an accident occurring because after all an accident at AT&T is only going to affect AT&T Verizon is not going to be affected an AT&T you know service technician or whatever who mistakenly does the wrong like service update or something is not going to have an effect on Verizon's Network so that is consistent uh if Verizon is not affected so maybe now we can kind of see the end goal here what we're trying to do is figure out which hypothesis is has more yeses which which hypothesis has more evidence supporting it than anything else right likewise we also on the inverse trying to determine which hypothesis is the least likely which has the least amount of evidence stacked up against it so maybe we can use this process to add a hypothesis or maybe even remove a hypothesis right if we find out that one of our hypotheses has no evidence supporting it whatsoever if we've got one of these hypotheses that has NOS in every column then you know maybe that's not a super accurate hypothesis and maybe we can move on to something else now with that in mind if we stop here we don't really have a whole lot of indicators working for us right because every hypothesis has two yeses behind it right so we're we're not really starting to break out these hypotheses and figure out which one's more likely so when we have a situation like that what we need to to do and this is something we need to do constantly throughout this whole process is keep adding evidence so let's go ahead and add in a few more pertinent details so for instance the next pertinent detail we can add is that this incident occurred at the same time or around the same time as a balloon incident occurred right an unidentified um potentially hostile uh aircraft over American soil so this got uh released a couple of days after the incident but based on the timing this would have occurred around the same time so that piece of information is very important because what does that do it allows us to put a yes in the China column but NOS in the other two uh hypotheses columns this is because well if China were to launch a Cyber attack you know chances are they're probably going to want to conduct some kind of reconnaissance on it right that's a very good uh military deception tactic but it's also a great intelligence collection tactic you take out your adversary's uh main means of communication and you have a Recon asset ready and waiting so that that Rec reconnaissance asset can then collect the data on maybe the response or the backup systems right if everyone's cell phone stops working then who what what are you going to have to go to Radio or maybe other platforms that would have a lot of military value for a nation state like China that's why the first hypothesis the the China did it story gets a check in this box right that's why we say this is consistent with China doing it this is more evidence that supports hey maybe China was involved in some way right conversely the other two it doesn't really make any sense right well for one the United States is not going to need to conduct reconnaissance of its own capabilities right so you know the United States launching a Recon asset or a Chinese balloon if that is indeed what it was uh that wouldn't make any sense for the United States to do that right as incord Nation with a cellular outage because the United States already has we already have you know the NSA and other federal agencies already have Untold unlimited access to Cellular traffic and radio traffic and things like that so they wouldn't need that reconnaissance so that's a no in this box that a balloon being Airborne at the same time as a cellular outage that doesn't jive with uh the United States you know the false flag hypothesis right and finally the same thing with with the uh accident story right there doesn't really there's no correlation there there's no like reason why this would occur at the same time so again you know an AT&T customer or an employee is not going to have anything to do with a with a potential uh reconnaissance asset so that's an interesting one and we can start to see how we're starting to break these out right now let us return to our attempt to identify some bias with this system with this whole yes or no consistent or inconsistent system system well if you are taking notes let's try to identify some of the opportunities for bias so one of the very first ones that I kind of hinted at right at the beginning is this first block here the incident lasting most of the day is not consistent with an accident this is a very very big distinction it may not seem like it's that important but we have to remember what was how was my line of thinking different than the first two blocks with this block I made the assessment that it is not very likely that an accident would occur that would last the whole day and I arrived at that assessment using statistics right using the averages right so in this block I used an average not a capability is it possible for an accident to last the whole day yes it is possible is it likely no but it's possible so if you're going to fill this chart out you had better dang well make sure that you're assessing each one of these blocks from the same kind of perspective possibility or likelihood you can't mix and match and we'll come to that here at the end and you'll see why this kind of matters another uh point of contention if you were if you paying attention and and you identified this potential bias is the facts of the case uh occurring at the same time as the balloon incident you know um timing is really really challenging because this system here is it's supposed to kind of help us identify correlation versus causation I know a lot of people in today's world tend to mess this the whole correlation versus causation up and they tend to slap that little Banner on things uh that they don't agree with because they're trying to make you think that you're thinking in directly but when it comes to timing of an incident when it comes to the uh you know hour of an incident or the day something occurs it's really challenging to put this kind of thing on the chart because it could just easily be uh something else right now this isn't so bad in this specific example because this is a pertinent fact right this is something that is very very pertinent and I think I think that it's a strong indicator right it's not a weak indicator it's a strong indicator of something right so this is one of those cases where the correlation being so high means that its importance on the chart is pretty is pretty good so um this is a point of personal bias though uh but again you know in the social sciences we can't remove all biases but we can reduce them quite a bit so let's clean off our chart here and let's let's go back to the very beginning and I mentioned that there's a slightly better way of doing this and it's sort of along the lines uh for those of you who are fans of the Carver technique it's sort of along those lines instead of saying yes no or maybe in each block we can assign a number right we don't have to assign yes or no plus or minus consistent or inconsistent we can give it an actual rating so let's go back and let's on a scale of one to five with one being the least consistent and five being the most consistent let's redo our whole chart here and again this is going to be largely subjective but it kind of helps us be more objective right so right here on the first one I'm not going to go down all of these but the very first row is kind of interesting because we have all threes right it's in there's an even chance either way that it's consistent or inconsistent with China uh doing it with the United States doing it or or an accident so what does this kind of tell us just as a brief side note when we have the same number in all of the columns this means that that evidence is worthless it doesn't help us break out a likelihood it doesn't help us develop this out so it really doesn't help at all now we'll keep it on the chart because it's a it's a fact of the case but you know it kind of equalizes everything now just going down the chart here we see you know some fives some ones uh some twos and threes here and there but you can kind of see where we're going with this and instead of adding up the yeses and NOS for each column our goal is to add up the total in each column so how many points if you want to call it that how many points does your hypothesis have and if we add all of these up we get a pretty interesting number don't we we get a 31 for the China did it Theory we have a 23 for the United States did it Theory and we have a 22 for the accident Theory so this kind of helps us you know now we can feel a little bit more confident in saying you know what considering all the factors you know the evidence is stacking up that maybe maybe China did do it does this mean that China did do it no not really um but it does uh really suggest that and what I want to draw your attention to if you didn't notice it when I put it up on the screen is this row right here having the knowledge of hindsight you know after the incident we have a statement by AT&T that it was an accident that's what AT&T claims okay now look at the values that I assigned to that I assigned a two a two and a five why is that why not ones well because of one simple fact AT&T could be lying a AT&T does have the incentive to lie to their customer base in the event of a legitimate Cyber attack uh so even if China did attack uh AT&T AT&T would have the have the motivation to lie uh and say that it wasn't a Cyber attack it was just a simple um misunderstanding or some kind of simple mistake so this is one of those things where so if you turn on your favorite you know uh mainstream media of choice and you see the statement that flashes across the screen in nice big letters AT&T says it was an accident you're likely to just sit down and say oh well there we go we're all done uh no no analysis needed I'm just going to pack up my stuff and go right there's there's no need to be worried about this at all but if you factor that in and you do some actual analysis and you work through some of these analytical techniques you find out that even AT&T saying that it was an accident is not a strong indicator so this comes into play a lot when it comes to some of these events right you might have people tempted to say well no you're wrong it's not a spy balloon it's something else oh no it's not a Cyber attack it's something else oh no it's not a political targeting event it's something else because look they said so they said it was an accident it was perfectly innocent haha all fine right well if you do the analysis and you can have the data on your side to say no it's it's something else right and though it's really challenging to arrive at this chart here even though this is what our numbers indicate we always have to remember analytical discretion never forget that at the end of the day your analytical discretion comes into play and that can Trump everything right you can have all these numbers lining up but if your gut is telling you something else after you've worked on this issue for a long time then you might want to listen to that and and apply a a more heavy weight to your gut feelings on things because that does come into play a lot more than all this academic mumbo jumbo sometimes so with all of that in mind let's Identify some of the disadvantages of this system I know we've identified quite a few along the way but let's kind of wrap it up and put a nice bow on it uh so that we can figure out if this is valuable uh for us one of the main disadvantages of this system is kind of tied into something that's kind of going on on social media right now and the public Consciousness is becoming aware of the very old um programming and statistics kind of theory which is that the purpose of a system is what it does not what it was designed to do or what it claims to do it is what it does so this system what does it claim to do the system claims to allow us to examine multiple hypotheses in such a way that we're not really able to cherry-pick data to support our own point it allows us to be objective and fair when assigning you know the facts of the case to our hypothesis the problem with this is that as you can imagine you can just leave out facts you can leave out facts that don't support your hypothesis or the one that you secretly or maybe subconsciously uh want to be true so if I was absolutely dead set on this being a false flag attack well I could just leave out you know the balloon incident I could leave out a couple of other indicators and oh look magically all the points go to my hypothesis do you see how you can still do that even though this process claims to not right this disadvantage is why in my opinion this is best uh assigned as a group project if you are sitting at your desk by yourself assigning numbers to these things it would be probably a good idea to bounce these numbers off of someone else and see what they think do your own Matrix and have someone else do the same Matrix and then aage your numbers together and that might be a better way of doing it right if nothing else you can go through the effort and then identify some of the variables and some of the thoughts that maybe you hadn't thought of before another disadvantage and this is a super super critical thing to remember that people really don't remember and it's very challenging when you're just number crunching and you're going through and you're just grinding away at this chart with you know 50 different pieces of evidence or factors it is really really easy to forget that you have to pick you have to pick whether or not you're going to assign this data this number to a hypothesis based on possibility and also based on likelihood can an adversary do this can evidence One support hypothesis one or can it not is it impossible for evidence one to affect your first hypothesis or is it likely right is evidence one likely to support hypothesis one or is it unlikely you can't just do well you know it's technically possible on the first hypothesis it's technically possible on the second hypothesis but it's not likely on the third hypothesis do you see the change in wording there and the change in mentality it's very very important for you to do both now you might want to do one Matrix based off of possibility like is it technically possible and you might want to do another Matrix based on likelihood and Compare the numbers right this is all what this is about it's just getting a good flavor and a good indicator of if you're on the right track or not it really does help you to rule out some more off-the-wall hypotheses sometimes a third disadvantage is that this takes a lot of time and as we all know uh in really any kind of world where you have to work on a project work expands to fill the time allotted if I have 15 minutes to provide an assessment on something then maybe I'm just going to go with my gut instinct rather than going through the mathematical process of drawing this chart and all that kind of stuff but if I have three weeks to get to an assessment well I can run through multiple um you know a matrices I can go through and I can do this many many times I can involve other people I can involve other factors as well I can remove hypothesis I can add hypothesis as new data comes in and I can really drill down get a pretty good analysis right but if I don't have the time to do that then you know my my numbers are going to be slightly off so my numbers if I were to do this in 5 minutes are going to be different than if my than my numbers if I had three weeks to work on this situation and that kind of ties into the fourth problem which is that um you really need a lot of evidence to have any sort of accuracy and you need a lot of hypotheses as well and sometimes the hypotheses are going to be so similar that that they're they're just different enough to say that they're different but they're going to be very very similar and when you have very similar hypothesis when the hypotheses are very very close uh in their their nature the numbers aren't going to help you very much so you might find that you know of course bringing back up our our own data here so based on the numbers we have you know based on the chart that we've done so far uh our accident theory is less likely to be accurate than the false flag Theory but it's only by one point right so that's not a very strong indicator that's a very weak indicator so uh I think that it's interesting and it's telling because based on this based on this chart here you can do a little bit of word smithing uh and if you're kind of a more nefarious uh uh person and you're trying to make this seem more uh significant than it really is you could say look based on my data an accident is the least likely scenario well it technically it is least likely based on this chart here but it's not a strong correlation right so you know that's just something to keep in mind is that you need more data and something to consider just as a brief caveat this is just the example that I have if you flip if you Google you know anal analysis of competing hypotheses and you find textbooks you find white papers you find you know other people talking about this very well-known Theory you're going to have not that many examples they're probably going to have a chart very similar to what I have here no more than you know maybe a dozen or so pieces of evidence to consider well if you want to get the most out of this you're going to have to do this with hundreds of lines of evidence right and a lot of people just don't have the time for that so you know again that's kind of a disadvantage of this system and the last disadvantage that I wanted to talk about is that you know if you're if you're correlations are weak if you do this whole process even with the numbers and you get to the end and you find that all of your data is just a couple of points apart well what does that tell you about your process I think that that tells you that your hypotheses are probably not right and that you probably need to bring in more hypotheses you need to bring in another you know one or two uh examples and scenarios that might be a little bit more accurate so this is why the a concept can be pretty pretty frustrating sometimes because you might spend an hour or two you might have your whole team you know you might have a team broken out you have three different dudes working on three different matrices and you come back and they're all either wildly different or they're all identical you know neither of which is very very helpful right because if they're wildly different that shows that you're probably not on the right track and this process didn't help you get on the right track and that's what this process was supposed to do likewise if you have uh everyone come back and you do this multiple times and everybody has EX exactly the same data that you have or something you know roughly very very similar well you know we're really at the edges of you know the fringes of what is helpful then because if you're all thinking alike then you're probably wrong about something so maybe it's time at that point to reexamine the evidence re-examine your hypotheses and go from there maybe use a different analytical technique and there's plenty of others that we can talk about that will be very help helpful but this is just one way of comparing differing hypotheses based on the evidence right it's very simple very easy to do you don't need to be an analyst or have any kind of professional training to do this and is very helpful for being a little bit more analytically minded in a world where you know fifth generation Warfare or information war is is just running rampant right so hopefully this has been helpful for some of you uh thank you all for watching and we will see you next time and as always fight in the shade