Probability spaces and random variables

probability is a tremendous practical importance but it's critical to make sure that we start on a firm mathematical footing from a philosophical and mathematical point of view probability is a pretty deep pool to swim in so we're just going to hit two high points the two core concepts we're going to talk about here are probability spaces and random variables we construct a probability space in order to formally reason about the kind of asymptotic frequencies of different outcomes there are three main ingredients and for that reason probability spaces are sometimes referred to as probability triples the first ingredient is a sample space which were denoting here as Omega and that's the set of all possible things that can happen it can be a finite or infinite set and when it's an infinite set it can be countable or uncountable the second ingredient is what we call the event space denoted here as calligraphic a this is a set of sets from the sample space Omega that seems like kind of a weird and confusing concept and it isn't really necessary to think about it for the finite case but it makes a lot more sense when we talk about uncountable sets the final ingredient is what we call a probability measure and it's a function that takes members of the event space a and turns those into numbers between 0 & 1 that we interpret as probabilities let's start out by talking about the concept of a sample space Omega one example of a finite sample space is the set of all possible outcomes of rolling two six-sided dice one example of an uncountably infinite sample space is the set of all points along the edge of a unit circle now let's talk about event spaces which are a bit subtle this is a set of subsets of omega event spaces have some pretty specific requirements that causes them to be a sigma-algebra now you don't need to worry too much about the concept of the Sigma algebra except to know that these properties need to be maintained two critically important subsets of Omega are the empty set and all of Omega itself no matter what both of these sets must appear in the event space a beyond that if we include any other subsets of Omega and a then we also have to include its complement the event space a must also be closed under a union that is if I take any two sets that are already an A then their Union also needs to appear in a being closed under complement and closed under Union implies that it's also closed under intersection let's look at a simple example with a finite sample space consisting of the six outcomes of a six-sided die to start building our event space a we must of course include all of Omega and we have to include the empty set now let's imagine that we want to include another subset here the outcomes one two and three denoted alpha then in order to satisfy our closure properties we have to include the complement of alpha note that both the union and the intersection of alpha and alpha complement are already in the event space now let's grab another event this time let's call it beta and make it just the outcome 1 as before we need to include its complement the set 2 3 4 5 6 let's denote these beta and beta complement respectively here we might think of an alpha complement and beta complement as being sets that were generated by alpha and beta now we need to also generate all the sets arising from the different unions the union of alpha and beta is just alpha and so that's already in the set but we do need the union of beta and alpha complement and then of course we also need the complement of that set which is just 2 & 3 we might say that this is the Sigma algebra or event space that is generated by Alpha and Beta that is it's the smallest such event space that contains alpha and beta thinking rigorously about event spaces in sigma-algebra is for the real line and other continuous spaces turns out to be a little bit more involved the mathematical area of measure theory is in part motivated by these kinds of problems we're not going to do any measure Theory here and we're going to assume that your intuitive notions of integration and basically work out if our sample space is something like the real line then the main thing we need from our event space is for it to contain all of the intervals that we might encounter as a result the event space we tend to like for continuous sample spaces is the one generated from all of the open subsets of that sample space this gives us an object that sometimes called a Borel sigma-algebra for our purposes you don't really need to worry about this level of detail but it's good to know what somebody's talking about when they use those words now the last piece of our probability space puzzle is the probability measure itself now this is just a function that takes members of the event space and assigns them a size that's between 0 & 1 and that notion of size needs to satisfy some properties are kind of intuitive if it satisfies these then we can call our measure a probability measure and the sizes are the probabilities of those events now every event space contains the empty set and all of Omega and those need to have measure 0 and 1 respectively the other property that we need is that if we have two disjoint sets that is two sets whose intersection is the empty set then the probability measure of their union must be equal to the sum of the two probability measures taken separately note that their union is guaranteed to be in the event space due to the closure properties we require now for a finite sample space it's pretty typical to take the event space to be the power set of the sample space that is the set of all subsets of Omega then we can specify the probability measure completely just by assigning a probability to each of the members of Omega that's the thing we usually call the probability mass function and of course it has the property so if we sum the probability associated with each member of Omega then we get 1 the situation in a continuous sample space is of course a bit more complicated here we need some kind of function that can take any well-behaved subset of Omega and produce a number between 0 and 1 here I'm showing an example of such a measure as I take a bunch of intervals along the number line and produce a number between 0 and 1 behind the scenes here I'm just using an arbitrarily chosen cumulative distribution function now that we've constructed a probability space we can finally construct a random variable the name random variable is terrible because what a random variable is is a function from Omega to another space here denoted calligraphic T if we go back to the case where Omega is the set of all possible outcomes of two six-sided dice then one random variable we might define is the sum of the dice so here the set T is just the set of integers between 2 and 12 the way we connect random variables to probability is we think about the preimage of any subset of T so here's a cartoon illustration in which we have a set on the left Omega and a set on the right T and the function X that is the random variable maps from Omega 2 T now if there's some subset s of T whose probability we want to reason about then we it's preimage in omega under the map X that is we look at the subset of Omega that if you applied X would lead to the set s we denote this as the function X inverse so then when we're talking about the probability of the set s what we're talking about is actually the probability of the preimage in omega and we tend to take it for granted that the preimage is a member of the event space that is that it's a measurable set now I know this feels like a lot of machinery and to be totally honest most of these things don't come up very often when you're doing machine learning in practice nevertheless just like many other things it's good with probability to have an idea of what's going on under the hood

Channel: Intelligent Systems Lab

Views: 11,221

Rating: 4.9457994 out of 5

Keywords:

Id: DqGUwoz4d4M

Length: 7min 2sec (422 seconds)

Published: Wed Apr 01 2020