Bayes Theorem and some of the mysteries it has solved

this video was sponsored by curiosity stream hold over 2000 documentaries and nonfiction titles for curious minds in 2014 a man named John Aldridge was working on a lobster boat he was 65 kilometers off the tip of Long Island and it was the middle of the night but while working a handle that he was using for leverage snapped sending him flying backwards into the ocean he yelled back to the two other people on the boat but they were sleeping and the engine was too loud so the boat which was on autopilot faded into the night it was 3:30 in the morning and John was stranded in the Atlantic Ocean with one goal to stay alive it wasn't until 6 a.m. when the other two men on board woke up and they immediately called the Coast Guard saying they've got a man overboard now they had no idea if he fell off five minutes ago or eight hours ago all they knew was their current location and the last time they saw him based on water temperature and weather the Coast Guard determined that John had about a 19 hour survival window before hypothermia took over and his muscles gave out so now it was a race against the clock to figure out where he was and here's what they did based on the boats current position and two autopilot speed they could determine the maximum distance John could be from that location from this we can sweep out a circle which tells us if John stayed put he has to be somewhere in that circle but the boat was on a course away from the shore so we could really narrow that region down then the Coast Guard input all the given information into something knows the search and rescue optimal planning system which operates based on math statistics and actual data about the weather ocean currents and so on the system superimposes a grid over the ocean and assigns each square a probability that John is there based on given information since there was a several hour long period where he could have fallen out we expect to see a long strip of high probabilities and thus a search began now if we first search here let's say and don't find him does that mean he's not there and thus the new probability is zero the answer is no he's a small head in a huge ocean we could just have missed them so after we search a grid we need to update its probability not just set it to zero and this brings us to Bayesian search theory an application of Bayesian statistics that's used to find lost objects the foundation - this is obviously bayes theorem but I'm just going to use conditional probability to keep things simple either way the goal here is to find the probability of some event a being true given that another event B is true for example if I roll the die and hid the result from you then ask what's the chances of five you would obviously say one in six but if I gave you more info like I told you the rule came out odd this is a relevant given that reveals more about the situation so the probability of it being a five given that it's odd is one-third new information can really change things now base theorem for the probability of a given B is written like this which can be derived from this formula which I'll be using soon and by the way this numerator just means the probability that both events are true now going back let's say we search this Square and don't find him we now want a new probability the probability that he is there given that we looked and didn't find him well for that two square he's either there or he's not maybe this Square has a 5% chance that he's there according to the software if he's in that square he will either be found or not maybe the skies are clear so there's a 70% chance that you find him if you look but still a 30% chance you miss him if he's not in the square then of course he won't be found with 100% certainty from the formula we saw earlier we are first going to find the probability that he is in the square and also not found as those are the two events I listed up here the probability that he's in the square and not found can be determined by just going down these branches and multiplying those probabilities together then we need to find the probability that he's not in that square period which can be determined by going down these two branches that both end and being not found we add the product of those probabilities and we get a value of about 1.5 5% which again is the chance that he is still in that Square given that we couldn't find him there if we go back to our grid that Square we just searched is now updated to its new 1.5 5% probability now since that square went down the other squares probabilities go up by a small amount it's obviously best to continue our search probably with this square but a few hours of searching we can see it might be more beneficial to search that original square again rather than one of these lower probability green squares out here now through various studies a while back it was determined that the probability of detecting an object is inversely related to the cube of the distance away or Alisa serves as a strong estimate but no typically more general detection equations are used the constant is usually determined experimentally based on things like depth of the water if the object is submerged or the size of the object that you're looking for this is one way that probability value we saw earlier could be estimated for those who are curious so if John is here let's say and a helicopter flies this straight-line path looking for him at any given point there's some probability of locating John based on the distance this probability changes at each point and if we integrate or sum the values along the line we get a total probability of detection for that line but we don't know where John is so we need to just do sweeps that optimize our odds which means we need to apply optimized search patterns now if there's a high probability of John being in one certain location like if we knew exactly where he fell off then an expanding square search might be ideal to cover areas of higher probabilities sometimes a creeping line search is better but in this case it was best to use a parallel search running parallel to the strip of high probability and yes that software we saw earlier was calculating all of this using math and statistics based on given information now after hours in the water John managed to find something to keep him afloat but he had lost so much energy I was preparing for the real chance of dying he decided to soon tie his body to the object so when he was found the body could be delivered to his parents they'd have something to bury but finally a three pm on one of the final sweeps the search team saw John and got him safely to the hospital where he was treated for hypothermia and made a full recovery so he could tell this story without John's ability to stay calm at sea and a formula that dates back a few hundred years it's very likely this man wouldn't be alive today next up in 1966 a United States Air Force bomber collided with another aircraft during a mid-air refueling over the Mediterranean Sea near Spain on board the bomber were four hydrogen bombs that fell to the earth three were recovered within 24 hours but no one knew where the fourth one was and thus analysts assumed it was resting somewhere at the bottom of the ocean to narrow down the search some experts put Bayesian search theory to work again the factors here were different than the last story though like the missing object was submerged in the ocean this time however in this case a local fisherman witnessed something fall into the ocean by parachute on the day of the crash which gave the government an idea of where the bomb likely was which really helped shape the initial probability map now two months had gone by with no results but remember failed searches lead to higher probabilities of other squares as things are updated after around 80 days the bomb was found almost 3,000 feet underwater and within 1.5 kilometers of where the fisherman predicted it's assumed that the fishermen's testimony saved the government over a year of work next up on June 1st 2009 Air France flight 447 mysteriously disappeared over the Atlantic Ocean as was traveling from Brazil to France and no one knew what caused it within a few days floating debris in a few bodies were found fairly close to the last-known location of the plane but in order to determine what caused the crash they need to find the black box or the flight recorders first they determine the maximum distance the plane could have flown from its last known position they expected to find the wreckage somewhere in this area then based on ocean currents and bodies that were found experts performed a backwards drift to produce trajectories that could locate the crash site they then found a 95% confident sone or a rectangle that likely contained the wreck location after several unsuccessful searches or research consultancy was tasked with creating a probability map using available information this was the initial distribution but given failed searches they were constantly making more updated maps and by the way I really can't do justice for all the analysis that went into this but soon after resuming the search using the statistical analysis sonar scans discovered large portions of debris and a few weeks later the flight recorders were discovered all right around the areas of high probability they found out that the crash was caused by inconsistencies with airspeed measurements likely due to tubes being obstructed by ice crystals the plane ended up stalling and did not recover and 228 people died on this flight many bodies were recovered and returned to the families to Barry however 74 were never found two years after this in 1968 for submarines mysteriously disappeared all which occurred within a few months period the French submarine has never been found the Israeli submarine wasn't found for 30 years and the Soviet submarine was partially recovered six years later but the USS scorpion of the US Navy was found just a few months after disappeared and although I'm not going to go into detail on this one it was the same Bayesian search theory that helped locate the wreckage now to be fair in acoustics Expert used sonar technology to help pinpoint where to look here which was a huge factor of course so I'm not saying statistics and probability were the only reasons these mysteries were solved but they did play a big role unfortunately when the submarine went down it took 99 people with it killing all of them and the cause of what sank the USS scorpion is still unknown to this day then in 1857 the SS Central America sank Turing a hurricane taking 425 of the 578 passengers and 14,000 kilograms of gold to the bottom of the ocean the modern-day equivalent of that is 292 million dollars by the way the location of the gold remained a mystery for over 100 years but in 1988 a man named Tommy Gregory Thompson used route information and documents from the time of the disaster to reconstruct the ship's path he then created a probability map of likely places where the gold would be located and despite skeptics that hard work paid off when he and his team recovered over 100 million dollars worth of gold so for anyone who doesn't see the use of stats it made this guy a millionaire but then he sold millions of dollars of gold before paying his investors and his team he was then sued by several of those people he went into hiding in 2012 and was arrested three years later and in November 2018 a jury awarded several million dollars back to investors and team members well this guy's so stupid I'll hold up can we just acknowledge all the dates I just said here because this gold was lost in 1857 before the American Civil War it was discovered in 1988 just a few years before I was born and that core case I just mentioned happened in November of 2018 which is four months ago as of releasing this video this is real recent and I just find it crazy that one storm that happened over 150 years ago is still having effect till this day okay now let's get back to the next story now moving on to a new category in 1787 and 1788 Alexander Hamilton James Madison and John J anonymously wrote 85 essays known as the Federalist Papers which were meant to persuade people to ratify the Constitution the specific author of each of these essays was mostly known with Hamilton and Madison writing majority however 12 were left for debate as no one knew whether Hamilton or Madison wrote them Hamilton was shot by Aaron Burr and died in 1804 taking the secret to his grave this remained a mystery for about a hundred and seventy-five years until two statisticians decided to take a crack at it they took articles in which they knew whether Hamilton or Madison was the author and analyzed certain words that one used more than the other the writing styles were extremely similar but words like while whilst upon and so on were a few words of interest since for example Madison rarely used the word a pond whereas Hamilton did in fact throughout 49 of Hamilton's papers they found that the word a pond came up 3.2 4 times per 1,000 words on average after analyzing all the known papers of both Madison and Hamilton a comparison could be made showing a big difference when faced with a mystery article the statisticians first assumed 50/50 odds that was written by either author they would then analyze the frequency of 30 words of interest and update the probability using Bayesian statistics nu given information like how often the word would appear allowed for a more updated and accurate probability of who wrote it now they first ran this test on known papers of Hamilton and Madison and it predicted the author correctly every single time and not just by a little the most inconclusive result occurred when running the numbers on an article written by Hamilton and it still said there was a 95 percent chance that he was the author so the tests seemed accurate and when it finally came time to test the 12 mystery articles the results that every single one was written by Madison the weakest case was number 55 which still said there was a ninety nine point six percent chance Madison wrote it is this a guarantee no even if the word frequencies were accurate it's possible the two edited each other's papers or something like that but we're definitely more confident than ever that Madison is the true author of those twelve papers then I'm sure many of you recognize this this is one scene from the movie the imitation game about how mathematicians broke encrypted German messages during World War two in the movie you see this electromechanical machine that tests possible real arrangements of an enigma which the Nazis used to encrypt messages due to the extremely large number of possible settings Alan Turing one of the inventors of the machine we just saw first need to drastically reduce the number of tests this machine had to conduct Turing developed a manual method to do this in which he would guess strings of letters in an unencrypted message which he could do because things like the weather report or phrases like Heil Hitler were repetitive and predictable he would then measure the validity of his guesses using Bayesian like methods updating the probabilities as more information came in in fact while doing analysis one of his co-workers asked him aren't you essentially using to which he replied I suppose so it wasn't exactly Bayes theorem like we know it but most people say this analysis involved Beijing and firms now there's of course way more to this and actually numberphile has a few great videos on how the codebreakers would slide the expected word along a string of encrypted letters which help reduce possibilities and I'll link that below but as we now know the analysis developed by Alan Turing and his co-workers is assumed to have ended the war several years earlier and the saved millions of lives again probability and statistics weren't the only factors of course but without them who knows how things would have turned out and by the way Alan Turing was not the only person involved in code breaking during World War two in fact a mathematician named Bill Tut who you've probably never heard of pulled off what some described as one of the greatest intellectual feats of World War two also helping save millions of lives I unfortunately don't have time to go over it but if you want to learn more than I highly recommend heading on over to curiosity stream who I'd like to thank for sponsoring this video curiosity stream is a streaming service that hosts thousands of documentaries and nonfiction titles spanning history physics in the universe technology nature and plenty more if for example you really want to change your understanding of the physical world some of my personal favorite documentaries to watch are those on quantum physics since pretty much every moment of these goes against everything you assume about physics founded by John Hendricks aka the founder of the Discovery Channel curiosity stream is an extremely affordable streaming service at only $2.99 a month they'll satisfy anyone with a strong desire to learn explore and understand the world around us the service is available on a variety of platforms worldwide including Roku Android iOS Xbox one Apple TV and more if you go to curiosity stream home slash major prep or click the link below and use the promo code major prep you'll get your first month's membership completely free and this gives you unlimited access to top documentaries and nonfiction series that I know many of you will find very interesting again links are below and with that I'm gonna end that video there you guys enjoyed be sure to like him subscribe don't forget to follow me on Twitter and join the Midd Facebook group for updates on everything hit that bell if you're not being notified I'll see you all in the next video

Channel: Zach Star

Views: 463,956

Keywords: majorprep, major prep, mysteries, statistics, bayes' theorem, bayes theorem, bayesian statistics, probability, mysteries solved with statistics, unsolved mysteries, mysteries solved with probability, applied statistics, applied probability, applications of statistics, statisticians, missing planes, missing submarines, air france flight, uss scorpion, how mysteries are solved, solving mysteries with math, solving mysteries with mathematics

Id: 82q3uYw6MuY

Length: 16min 17sec (977 seconds)

Published: Mon Mar 11 2019