Defeating Deception: Source Reliability

hello again everyone and welcome back to the underground these days it seems like information is flowing in every direction all at once and most of it seems like it's fake right as we move into more of a fifth generation Warfare battle space and as the power of AI becomes even more of a concern a lot of average people out there are going to have to take an interest in determining what's real versus what's fake and this has become a lot harder to do as the information sources have become more WID spread making sense of the world around us today is very tiring and exhausting work so much so that I really wouldn't fault anyone for thinking that everything is fake and this is all just a lie right so clearly the average person has their work cut out for them if they want to separate what's real from what's fake obviously this is a very huge topic but today we're going to kick off what is I think going to be quite a long series on defeating deception separating the real from the fake I have really struggled to figure out how to best convey this topic so rather than bite off more than we can chew today I think I'll start with Source reliability Source reliability or the source reliability Matrix is really a way of grading human sources we have to understand how the grading process works before we can get more into how to develop more human intelligence sources so right up front what is human human intelligence is intelligence that is derived from Human sources so this is a pretty big field of study as we can see human or human intelligence is also the oldest intelligence discipline going back thousands of years to the Biblical times right where we have spies and deception and Espionage being conducted even before the United States was a thing today we have many things that we can talk about in the field of human and many Lessons Learned along the way but one good place to start a little tiny nug of information to start with is the source reliability Matrix and here's what this is it's very simple it's a two-part system comprised of a letter rating and a number rating when put together these give us the reliability for a particular human source as you might imagine hum is a very subjective field right whereas things like Sant and IMT these are objective things right a camera sensor it doesn't lie you know the analyst who interprets the photo May lie but the camera sensor itself is going to tell the truth it's going to give you that raw information well with humans we don't have that luxury humans are always going to be handicapped by the ability to see things through their lens through their perspective everything from culture to language to motivations are these are all going to factor in to a human intelligence source and spoiler alert it turns out people lie kind of a lot so a great rating scale was developed in order to kind of rack and stack these intelligent sources and hopefully allow us to see which sources are better than others so we have our two charts we have the letter scale and we have the number scale these get again combined together to give us our final rating the letters describe how reliable a person is the person themselves how reliable they generally are and the number scale designates how credible that information normally is this is very simple like it it can be overthought quite a bit as a matter of fact I myself have really really tried to think about this process over the years but this is the simplest it can be so using these charts if I were to tell you I have an A1 Source this means that the source is extremely reliable and the information that they give us is extremely credible again this is not something that's very common in the human world most of the time we're going to get b or lower uh levels of uh reliability and two or three uh levels of uh information credibility generally speaking I this is very controversial and I I personally don't agree with it because this is one of those things where there's more exceptions than than the actual rule itself but in theory you should be able to say that an A1 source is better than a like D3 Source or something like that now some of you may have noted that this is not quite right there there is actually a huge debate about this indicator right here the number rating scale so let me break that down a little bit because it it may make things a little bit easier because even in the Intel world there's a lot of confusion around this and believe it or not there are contradictor Doctrine points so so the number scale is actually most often applied as an average it shouldn't be but it is if you pull open your handy dandy human manual and flip to the actual page that describes this you will find that the number part of this rating is supposed to apply to the actual specific human report itself I know this is really really confusing but there are a couple of different ways that people tend to look at these Source reliability ratings some people think that both the letter and the number are an average doctrinally speaking for without question the the letter rating is an average but the number side of the equation can actually be referred to a specific information report this really matters a lot I think for the prepared citizenry space because if you're trying to do like some ENT or open source intelligence yourself and you come across like some primary sources on say you know social media or something it's going to be awful tempting to slap that like A1 or A2 or B2 or B1 uh label on the whole account when really you need to be putting that lab on each individual report again in the real Intel world as far as my own professional career kind of indicated people don't tend to do this so much you're going to see different like reports like human reports that have different you know ratings on them usually you're just going to get uh human intelligence reports where that human intelligence analyst is just going to rubber stamp that label on pretty much everything a source provides right corroboration is a huge part of this as well and we'll get to that in just a moment but for right now I wanted to point out the distinction between that that number uh side of the chart because that really is kind of a a sticking point even in the professional Community sometimes and that's really it one of the things though that you might realize and the next question that you might want to answer is okay well how do you give these sources these ratings seems pretty subjective for a system that's designed to be objective as possible right so we find that in the act of you know squaring that Circle we find that there's a lot of problems with this system again I can talk about this for many many hours but for now this is how the system works to answer the followon question of okay I understand the system but how how does this help me how can I how can I go through this and apply this because what most people are probably going to want to do is take this number system and apply it to like a journalist or an information Source or like a telegram Channel or a Twitter feed right someone's going to want to use this practically well this is where it gets kind of challenging the processes by which we assign these uh numbers and letters to a source is is pretty wide ranging now it shouldn't be but you also have to remember that we're having to deal with public information and just to cut right to the chase I know it's not the best way to teach it but this is the one of the more interesting uh research sources that I found on this topic so let's talk about that now one of the more interesting uh information sources that I have found on this topic that is probably a little bit more helpful for those who don't have a professional uh career in intelligence analysis is this white paper from the Canadian defense research and development center so this is a military Think Tank kind of thing and they put out this white paper which you can read for yourself the link will be of course in the description and this outlines not just the scale but some of the problems with it namely that this is not an evaluation methodology it's a scale it's a grade okay you know it doesn't really show us how we can evaluate sources even though it it kind of claims to so very helpfully this white paper gives us access to a couple of charts which are very helpful to have from rva's ontology of quality of information content quite a mouthful right but this is actually a section from a textbook uh or or rather more like a a research paper a really long research paper that was published by galeno rova in 2016 this is one of the more recent sources that's not American that I feel is probably quite interesting to read you can get the textbook yourself if you have Deep Pockets and want to buy a random obscure textbook but this free white paper kind of outlines the graphics that we need to know which are these these two charts help us list some of the attributes that we need to think about for giving that number rating and that letter rating to a source now working a little bit backwards let's start with the information side of things the quality of information content side of the chart this is that number scale again so really on this really complicated flowchart really the things that you need to think about when it comes to the quality of information and trying to give it that number is the accessibility of the information the timeliness it was given the Integrity of that information and their relevance of that information just running through these really quickly accessibility is the first thing you need to consider and this this is for two reasons one The Source has to have access to the information if they don't have access to the information then they can't possibly have given it to you right also accessibility needs to be on your end as well so you need to be able to have other access to that information to corroborate we don't want single Source reporting meaning there's one guy who's the only source for this information and that's it we have no other way of confirming this knowledge moving on to timeliness the information has to be actionable intelligence that is given late is history that's what it is intelligence that is not given in a timely fashion to be militarily effective is useless it might as well be in the pages of a history book Integrity we'll come back to in a second but relevance the information has to be relevant right if they're giving you Stacks and stacks and stacks of you know binders of information that you don't need then they're not really going to be a very useful source for you moving back to Integrity there's a couple of things you need to think about with that because the Integrity as you can see from this flowchart can be really really indepth but I'll just focus on the first uh first two boxes there uncertainty and imprecision all human beings have a bit of uncertainty to whatever they're doing so the information when it comes to whether or not it's true there's going to be a level of uncertainty no no matter what we get into the philosophical debate of what is truth really quickly but really you have to make sure that that information that you're given is subject to a little bit of uncertainty because if a source is too sure if they are too certain about something something's off because there's always a level of uncertainty right also imprecision this is probably a little bit more easily understandable by witness testimony if you've got a a crime that happened and you've got 10 witnesses to that crime and you interview all 10 Witnesses separately and all 10 Witnesses have exactly the same story something is up something is up because people never agree on uh details down to a a very precise degree so if there's a certain level of imprecision like your Source well I I don't know or they're slightly inaccurate with something then you know that's not necessarily an indicator of anything nefarious it it is an indicator of uh the imprecision factor so something to think about as well you don't want it to be too imprecise because then you're just wasting your time but you don't want it to also be too precise because that's an indicator of some prior planning and maybe some deception being used against you so in the interest of keeping things as short and sweet as possible this is the information side of things moving on to the actual human side of things we have to recall that human beings are subjective sources they're not objective usually however uh if we look at the left- hand side of this chart we we can talk about some things that we want to make sure our subjective sources are kind of thinking about and kind of privy to the first one is objectivity again humans are subjective they are not objective by their very nature but we want them to try to be objective we want humans to try to be as neutral as possible we want to examine any motivations that may sway that objectivity we also want to consider things like culture Heritage even language uh barriers which may bring in an element of uh subjectivity when we're trying to be as objective as possible and get just the raw data so that we can figure out what's going on we don't want our sources making up the story for us we want to figure out the story for ourselves so getting a source who is objective and just giving the raw facts just the facts ma'am you know that's what we want to do this to kind of help this out really this really really helps objectivity is if a person or a source is has a level of expertise that allows them to understand the information they are conveying think about it like this if you put a soldier on top of a hill and tell them you know look through these binoculars and tell me all the tanks you see they're probably going to give you a salute report or a salt report or some former fashion right some professional doctrinal product but you know even just me personally I get a lot of emails all the time of people saying oh my gosh two helicopters flew over my house today and then I asked them what what kind what kind of helicopters were they and they say I don't know I don't I don't know how to uh identify helicopters well you know I can't really do anything with that because it could be a news crew it could be a crop duster it could be a firefighting helicopter or it could be a Blackhawk I you know there's nothing that I can do with that so having a level of expertise to understand the information you're trying to convey this is going to be very important for a source this also helps us determine the next one which is truthfulness How likely is that source to be very truthful are they motivated to lie to us based on any of the prior identified factors very important to consider right again this is one of those things where it's like well you know we're trying this whole process is to determine truthfulness not it can't be a part of the process too so we just move on from that when and move on to reputation which I think is more accurate which is does this person have a good reputation or a poor reputation when it comes to reputation it's not simply about the reputation of the information they've provided but it also could could apply to their reputation within their institution are they well- known well-liked are they likely to be given further access to more information further on because they're just known for being a good worker and because they're known for being a good worker they can give more information because they're giv more access do you see how this snowballs into potentially more important human disciplines as we move along but for now keeping things simple the reputation of the source is important to consider finally confidence this kind of goes back to the level of expertise but how confident is the source in their own information we see this an awful lot in the Civil space even if we don't realize it because a lot of times we get reports from people who don't believe what their own eyes saw right they're just reporting something but because they don't believe what they saw they may ignore it you know like a lot of this comes into the comes into play with things like UFOs and you know Bigfoot and some of these larger more theoretical um things right you could literally have you know a team you know sitting on a mountain in Afghanistan and listening to the local lore right of all the the you know magical creatures that inhabit the the Hindu Kush mountains and if you're a soldier sitting on top of a Hilltop and you see some weird stuff at night you know some being or craft or something like that you may not be inclined to tell your S2 shop about it because you think you might have S seen something that was just way too weird when in Ence I may be sitting on a report that says oh the Russians are using some new technology across the way there and um I'm not going to tell you because you're going into the field you know you might get captured so uh something like that your source needs to have confidence in their abilities and their means to convey information to you so wrapping things up I know this is a lot of information to take in but this is this is a very large topic and this is the easiest way that I found to talk about it I've recorded this at least five times now so really this is just a way of saying look you're you're trying to use an objective rating scale to rate a source but the way that you get those numbers is pretty subjective based on a variety of factors that are outlined in this white paper there are other sources fear not all the human guys out there saying you left some stuff out don't worry I'm going to be covering a lot more of this in the future this is just a good way for like the average like you know Western population to kind of understand the the basics of the grading scale so we can talk about more complicated stuff later now again just as a caveat I want to make everyone very very very aware that this system isn't the greatest and we run into right up front like right up front we run into the issue of the purpose of a system is what it does not what it was designed to do but what it actually does so sometimes me the person who might be using hum reports and hum rating scales I may have two reports sitting on my desk one of them from an A1 source and one of them from a B2 Source now according to the scale the A1 source is better you know uh but I may find that they're two different report they're two different people they're two different sources two different levels of access two different all the stuff's on this chart right so you can't compare the two it's a system that's designed to take the to to compare apples to apples but it ends up you know comparing apples to Transmissions it you can't really compare subjective data and this system tries to do it and it doesn't really do it super well in my personal opinion I have had more A1 sources lie to me than I can count I have had more fake news fake information come from A1 sources than anyone else because the A1 Source because it's got that nice little steak sauce brand on it people tend to think that it's better and this is what I kind of want to end on is just be careful right when someone tells you they've got an A1 Source right that doesn't necessarily mean that it's the best and that doesn't necessarily mean that they're telling the truth again a lot of these factors come into play culture is probably one of the most pertinent ones that you could possibly imagine when it comes to the lens by which I'm looking at this which is through the lens of counterinsurgency in the Middle East over the past 10 20 years that's my perspective and as I saw it uh working in that career field for you know a few years I did not really see any good hum at all now I'm sure the human guys would disagree but personally I I think that a lot of us tended to forget that if you are an American stationed militarily in a Middle Eastern country the people in that country are not going to want to tell you the truth it's a cultural thing it's a uh sometimes it's a language thing but mostly cultur for mostly cultural reasons I think so that's just something to consider human in you know Eastern Europe I'm sure is quite different you know human in you know Western Europe quite different South America very different right but as I saw it human in the Middle East we should not be using that at all um and this is something to consider because that same kind of effect applies here in the United States as well but enough ranting on that I just wanted to show everyone The Source scale some things to think about if you're trying to rate your own human sources and kind of apply thisw logic to you know this new journalist you just found out about or this new like Twitter news source or this new telegram Channel you know if you're trying to figure out things this might be a good way to think about very good way to start but again there's a lot more to it so we'll talk more about that at a later date for now we know the scale and this is a good foundation to build on for next time so I know this has been really confusing but thank you all for watching and we will see you next time and as always fight in the shade